InfoQube IM - Community

Submitted by Armando on 2015/02/05 14:22

Sorry to come back again with this! And sorry to bother you with that after your hard drive crash, Pierre... but it's becoming very difficult for me to work :
IQ, even if the greatest PIM in the world, really slows me down these days.

Quite possible that users not using equations/autoassigns/conditional formatting don't experience those performance problem.

------

I don't know if the way equations/auto-assigns are managed has changed in the last few months (?), or if it's related to the new grid "refreshing" routines/events, but each time some change is made to an item -- and especially if there are some equations/auto-assigns involved, it's really, really slow. And by slow, I mean... s-l-o-w.

5-8s waiting here and there disrupts my thinking process. I could post a screen capture if necessary.

I tried working with only one grid, but it's not practical and the difference isn't that great. I'm not trying to send a spaceship to Mars here, I'm just marking items as done, adding dates through auto-assign equations and stuff like that!

Anybody else experiences slowness in specific tasks ? Descriptions could help isolating the problem(s).

Tests would have to be made with a relatively large DB (to be somewhat realistic...), a few open grids, fields with auto-assign equations + row/column equations, and conditional formatting.

I'd like to give more details, but I'm in the middle of moving + writing a thesis, so it makes it real hard. (Note that there are various threads in the forum where I detailed performance problems regarding auto assigns/equations)

Comments

Hi Armando,

Sorry to hear you are having performance issues. I have a 12 year old notebook PC (yes, I know, an antique it is), years of stuff in it and have no performance issues...

If you can give me the last version which did not have that issue, I'll be able to see what has changed.

p.s. You and I know that you have an "extremely" complex database and that what you now do with IQ, you could not do with any other piece of software...

Just spent 45 min timing and writing stuff, and I lost my post hitting backspace. The web page reverted to the previous page... More time down the drain.

Yes, as I said IQ is the best PIM, in terms of flexibility, etc. But it doesn't mean that performance can't be improved.

If I post these concerns, it's not to bash IQ, but because I hope that some user will recognize something and will be able to contribute some insight.

Anyway, this should give an idea.

This is in a Boolean field called FollowUp

auto-assign:

A:FollowUpDate=followupValue(TaskActStart, FollowUp, FollowupDate, taskStartDurLock, Rechour) |

The Function it refers to is:

Function followupValue(TaskActStart, FollowUp, FollowUpDate, locked, recHour)

Dim rightHour

If (Not IsNull(recHour) Or recHour > 0) Then rightHour = recHour Else rightHour = 0.25

If followUp = False Then

followupValue = Null

Exit Function

ElseIf locked = True And Not IsNull(followupdate) Then

followupValue = FollowUpDate

Exit Function

ElseIf (((TaskActStart >= Now) Or (rightHour >= Now-Int(Now))) And (Int(TaskActStart) - Int(Now) <= 0)) Or (IsNull(TaskActStart) And rightHour >= Now-Int(Now)) Then

followupValue = Int(Now) + (rightHour)

ElseIf (TaskActStart < Now Or (rightHour < Now-Int(Now))) Or (Int(TaskActStart) - Int(Now) > 0) Or (IsNull(TaskActStart) And rightHour < Now-Int(Now)) Then

followupValue = Int(Now) + 1 + (rightHour)

Else

followupValue = FollowUpDate

End If

End Function

Yes, it's ugly as hell. But, to my defense, it was written 7 years ago and it does what I want. Not sure if it could be tweaked so that it runs quicker. Maybe, but I don't think I'd gain a 50% performance increase from that work. Maybe...

Anyway just applying that auto-assign equation to 100 items takes 43s

Now add those auto assigns (as in my own DB):

A:itemforecolor="&h91CFFF" |

A:icon_statut=AddKeyword(icon_statut, "<img>hourglass_trans</img>") |

And the total is now : 64s (this takes into account the "refreshing", of course as it's not possible to do anything while IQ is refreshing the grid cells)

For 10 items it's 6.5 s.

IMO, it's slow.

Let's take a simpler case

fictive field : addNumToNum

auto-assign:

A:order = order + 3

Applying this to 10 items takes 2.4

Applying this to 50 items takes 10.5s

Applying this to 100 items takes 20.9s

It's still not super quick.

those are relatively simple examples as there are no row or column equations involved at the same time. Add rows and columns into the mix [Edit : actually, in my first example, there is a row equation involved in another field, and it doesn't help, of course] and...

[EDIT : I just want to make sure everybody understands that I do know that my equations etc. aren't super simple. But, at the same time, I'm wondering if 21 sec. to add "3" to 100 items is a normal speed. Is it ? If it is, then I'll just accept the situation, swallow the pill, and simplify all my DBs equations -- which I should do anyway.]

Thanks all.

-------------------------------------------------------

Windows 8.1

Sony Vaio S Series 13 (SVS131E21L)

Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

OK, I've tested it on a sample database.

On my "slow" PC the times are (for 100 items)

FollowUp auto-assign 8 seconds (you have 64)
addNumToNum: 4 seconds (you have 20.9)

I've enclosed the sample IQBase (using grid Inbox). Can you check what are your times with this file ?

[edit] Approx same times with my new Windows 8.1 tablet, an Acer Switch 10.

Not stellar, but the fan-less quad-core processor run at around 40C and consumes only 0.25W on "idle" and 0.75W when IQ is working 100%... amazing !!!

Thanks a lot for looking into that!

I'm only going to look/work at/with these numbers as they aren't affected by any row/column, equations :

fictive field : addNumToNum

auto-assign:

A:order = order + 3

Applying this to 100 items takes 20.9s

This is 5 times the times you get. I'm puzzled, but I don't have the time to explore more unfortunately (moving, urgent deadlines...)

Maybe it has to do with the number of columns ? The size of DB ? the number of grids ? the number of fields ? So many parameters.

-------------------------------------------------------

Windows 8.1

Sony Vaio S Series 13 (SVS131E21L)

Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

Try it on the sample app, we'll be able to have a better idea of the cause

addNumToNum

auto-assign:

A:order = order + 3

Applying this to 100 items :

On my DB, 1 grid with 4 columns : 8.6 sec

On my DB, 1 grid with 44 columns : 8.7 sec

with 5 other opened grids : 12.1

with 10 grids : 21.8

with 20 grids : 23.4

So, preliminary conclusions:

1- my DB *is 4x slower* (maybe because it's bigger? Number of fields? number of items?). Not that addNumToNum is an autonomous field, and that the order field is not linked to any row/column equations in other fields. However, there's some conditional formatting involved. Conditional formatting might affect performance -- will test later.

2- adding more grids to the mix seems to really affect performance. However, the difference between 1, 5 and 10 is big, but not between 10 and 20.

I don't have the time to do more tests right now.

I *never* work with less than 10 opened grids. The norm is actually closer to 15-20.

Simply because there are no ways to quickly classify, prioritize and open groups of grids in IQ.

Depending on what you're doing (not if your only managing a phone list, a task list, personal finances and a few simple projects... ) it's way easier to have your work right there than buried in 100s of other grids stacked in a list. So the only solution is to open them and put them in a separate tab group (& maybe "hide" the tab group once in a while, etc.).

[2015 02 06 -- 12 21 : edited post to add details]

-------------------------------------------------------

Windows 8.1

Sony Vaio S Series 13 (SVS131E21L)

Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

In v0.9.26PreRel40, I'll only update the current grid. This should speed things up significantly.

We'll see if it is penalizing in any way to do thing this way...

(quite a few apps work this way... to update a window, one must give it the focus)

Thanks Pierre. You answered as I was was writing my post (below)

----

Grrrr... Wondering why the numbers were so different from the sample, I found a "hidden" equation indirectly linked to "order". Removed it, presto.

Now the numbers are still a *bit* bigger, but not by that much; so the problem doesn't lie with the size of my DB (I'm relieved!!!!) nor with the amount of fields/grids (relieved again).

New numbers with the deleted extra equation

with 20 grids, 100 items : 14.7

with 10 grids, 100 items : 11

with 1 grids 100 items :5.1

removed conditional format totals:

with 20 grids, 100 items : 14.7

with 10 grids, 100 items : 10.2

with 1 grids 100 items : 5

OK. So I'm basically back to square one : the number of opened grids + row equations/auto-assign equations are what slows down everything the most.

-------------------------------------------------------

Windows 8.1

Sony Vaio S Series 13 (SVS131E21L)

Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

This is good news !

What do you think of my idea of only updating a grid once it receives the focus ?

Better still, I'll try to put the update in another thread, and adjust the update frequency based on whether the grid has focus or not.

Basically, non-focused grids will get updated, but at a slower rate

That approach seems good. The only potential problem is the background updating of other grids.

I wonder if IQ could do the updating ~~ether~~ both

- as you suggested, when the grid receives the focus

- as you suggested, in another thread, but only when IQ/the PC isn't in use; this would accelerate performance all around. Maybe it's not feasible though.

Updating the non-focused grids at a constant (lower) rate might work, but I'm not sure how much will this still affect IQ's operations as it's trying to keep up with all the queued changes.

[Edited my meaningless post]

-------------------------------------------------------

Windows 8.1

Sony Vaio S Series 13 (SVS131E21L)

Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

In v0.9.26PreRel40, the grid that has the focus is updated immediately. Other grids are updated in a different thread and should not slow down the UI. Fine tuning may be required... we'll see.

I hope this will be the best of both worlds...

Next up is optimizing the actual calculations and DB storage operations

Pierre_Admin

[quote=Pierre_Admin]

In v0.9.26PreRel40, the grid that has the focus is updated immediately. Other grids are updated in a different thread and should not slow down the UI. Fine tuning may be required... we'll see.

I hope this will be the best of both worlds...

Next up is optimizing the actual calculations and DB storage operations

Pierre_Admin

[/quote]

Seems very good to me!

As always, thanks Pierre.

-------------------------------------------------------

Windows 8.1

Sony Vaio S Series 13 (SVS131E21L)

Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

I've released the portable version now.

For the installable one, will have to wait till next week. The installer software is not yet installed on my PC... so much to do !

Thanks a lot! Big improvement.

As explained there : Version 0.9.26Pre-Rel40 is now available !

Using the same field + equations (and I also put back my extra auto-assign equation):

- Previous action on 100 items with 20 grids : 23.4 sec

- Now : 6.1 sec

So... in that situation that's almost 4x faster ! Your mileage may vary, of course. That said, even with only one grid, and no extra equation, it's faster : it was something like 2 sec.

-------------------------------------------------------

Windows 8.1

Sony Vaio S Series 13 (SVS131E21L)

Ram:8gb, CPU: Intel i5-3230M, 2.6ghz