InfoQube IM - Community

Submitted by lucasd on 2010/12/02 00:44

My IQ database has started to report a number of errors which I'm not sure how to interpret. This evening, it also auto-generated nearly four thousand blank items.

My database is still usable, and I can probably figure out how to iron out the kinks, but would it be helpful to post the error log?

(EDIT: I forgot to mention that I'm using 0.9.25H, but the problems began before switching to the new version.)

UPDATE: Now things are getting weirder. I tried to delete the nearly four thousand items. 85 of them persisted, and I tried to delete them multiple times. Finally I got rid of them, but then I discovered in another grid that there were exactly 85 blank items that I hadn't created. Whereas the four thousand blank items all had item ID's, these 85 blank items have no item ID's! And I can't delete them. Here's a screenshot showing the blank items with no ID's:

UPDATE 2: Now I am unable to open my database at all, but I should probably go to bed anyway. Will tackle it again in the morning.

Thanks,

Lucas

Comments

What is the one value that is shown (but not expanded) in the properties field (just to rule out a display issue)?

Have you tried a database repair?

Menu: Tools >Database Management >Repair

Have you regular backups?

Tom wrote:[quote]What is the one value that is shown (but not expanded) in the properties field (just to rule out a display issue)?[/quote]

Not sure, because I can't open the database, but I think it's for the Yes/No field for inclusion in the "@Today" grid.

Tom wrote:[quote]Have you tried a database repair?[/quote]

Yes, I tried that several times before the database became un-openable, but it didn't seem to solve the issue.

Tom wrote:[quote]Have you regular backups?[/quote]

I wouldn't say regular, but in this case I do have a back-up from before the problems became severe (although there were already some problems at that point).

Jon wrote:[quote]You indicate some problems before using "H." Were you having all these symptoms prior to upgrading?[/quote]

Good question --- I don't have an exact answer, but as far as I can tell the root of the problem began before upgrading. In fact, the main reason I upgraded was that I hope it would lead to fewer errors and less weird behaviour.

Armando wrote:[quote]A long time ago It happened to me because the Iditem number field wasn't protected in the import process... and some import process corrupted those and then my dB went havoc.[/quote]

Interesting... Actually, the weird behaviour didn't begin right after my last synch/import. Rather, it began later, after I did some more editing involving assigning items to other parents. In fact, about a month ago, something similar happened with the generation of hundreds of blank items, and I hadn't done any importing. Rather, it resulted from heavy-duty "cloning" (I know that's not the right word), involving all sorts of incestuous relationships between items.

Hi Lucas,

Seems like a possible Database corruption -- Iditem should always be there.**

In any case, make sure you have backups from before the problem and keep them safely.

Send Pierre a note describing the problem.

Good luck

**A long time ago It happened to me because the Iditem number field wasn't protected in the import process... and some import process corrupted those and then my dB went havoc.

The version you are using has not been posted yet. Are you testing "H" for Pierre on a limited basis? You indicate some problems before using "H." Were you having all these symptoms prior to upgrading? Sorry, but I cannot test.

Jon

The "Latest Build" version of the IQ exe only on the download page is either an urgent bug fix version of G or an early release of H depending on your point of view. Either way it shows a version of "H".

HTH

John ( <-- also a version with "H"!)

[quote=jdonlan]

John ( <-- also a version with "H"!)

[/quote]

I hadn't visited the download page since G1 was posted. The home page has not been updated, so I assumed Pierre was still working on H. Thanks for alerting me.

Jon

As Armando pointed out, ItemID should not be used in an import / sync process. I'll add the required protection.

Regarding your database, you have a number of options :

Use a pre-sync backup
Send it to me
Wait till v0.9.25H is out, where I'm adding a fix for this issue

The bottom line, is that it isn't strickly speaking database corruption, but that the ItemID field is not behaving correctly

HTH

[quote]As Armando pointed out, ItemID should not be used in an import / sync process. I'll add the required protection.

Regarding your database, you have a number of options :

Use a pre-sync backup
Send it to me
Wait till v0.9.25H is out, where I'm adding a fix for this issue

The bottom line, is that it isn't strickly speaking database corruption, but that the ItemID field is not behaving correctly[/quote]

Okay, but I did not use ItemID in a sync process, so there must be something else causing that field to misbehave.

That would be great if you could have a look to see what's wrong. Which email address should I use? Mine is a gmail address: "ldreier"

Thanks for you help, and thanks also to everyone else for the helpful comments above --- much appreciated.

>As Armando pointed out, ItemID should not be used in an import / sync process. I'll add the required protection.

Starting v0.9.25J, Import, Ecco Import and SQL Synch perform checks to ensure that the IDItem is not used in the process.

Note that the blank item generation is probably due to some kind of bad loop : IQ's source field auto assignation + manual settings in field auto-assignation seems to have the effect to auto generate items in certain cases (inheritance is on for a field and this same field is both automatically assigned by IQ on TLIs and manually set to be applied to all items created in a grid...). I'm not 100% sure, but I had a grid with that problem and since I corrected the set up in grid management, it seems to be okay. Will be easier to explain on the phone.

Hi Lucas, Were you able to solve that problem ?

hummmm...I just saw something that looks like that in my own DB. 6646 empty items (not completely empty though) were created between 2010-12-07 00:45:44 and 2010-12-07 21:55:32. I have no idea whatsoever of why this happened. Will keep you posted.

These blank items creation are linked to these symptoms : unresponsive DB, shut down DB, and then reloading the DB. - On the 2010-12-05, exactly at 01:33, my DB froze and I had to shut it down. When I reopened it, it opened slowly. Took a while to become responsive. 453 blank items got created at that time. Not completely empty (Grid "source" fields are there etc., but item field is empty and they're all created in a relatively narrow time frame). Here's the time frame this time for these blank items creation : 2010-12-05 01:34:07 -- 2010-12-05 01:38:58 - Now, on the 2010-12-07 at 00:45, same thing. IQ became unresponsive. Not sure what I was doing exactly. P.S. : Note that these items all appeared in the days following the last IQ update : 2010-12-03 00:53:56 - On the 2010-12-07 at 13:45, I had to shut down IQ as it had become unresponsive after I pasted a big piece of HTML in the HTML pane (know that from my log). I tried reloading IQ several times between 13:45 and 13:55 and it wouldn't. I finally deactivated the option to open the last opened grid, and it finally opened. Well... Guess what ? Precisely : Between 2010-12-07 13:45:58 and 2010-12-07 13:55:18, 1140 blank items got created. - On the 2010-12-07 between 21:52:06 and 21:55:32. Had to shut down IQ and restart it. Again, this is when I got another 866 new items... Unfortunately, not sure what I was doing exactly. I know I was reading an article on the web. Maybe I tried to clip it into IQ or something like that. From my log it's hard to tell. I don't know what's this all about... And this is all the details I can provide. For now.

I'm also having a bit of a problem deleting these empty items... Probably for a different reason than Lucas though. These are part of hierarchy with conditional calculation, and I have the feeling that their deletion is on the extremely slow side. It took more than 2 min. to delete 18 items... And I have about 6400 to delete. That means around 12 hours for all of them... I'll probably try disabling all equations etc. for the deletion process. :( I think it would be nice to have something like a "Quick delete" option which would bypass all equations. A kind of "reverse import" (I say that because when items are imported in the DB, no equations at all are applied and those are added extremely quickly -- food for thought).

Hmmm... When i had problems deleting the blank items, I think it was also because it was simply taking a very long time. In fact, I think that may be why the items had nothing in the "IDItem" field --- it may have just been taking a very long time for that field to get populated after the items were generated.

I never did solve the issue, by the way, but I just started that database over from scratch, since it had been a new database anyway. (I did send Pierre a couple versions to have a look at.)

Anyway, I hope you're able to troubleshoot this issue and keep the DB intact.

I have the feeling that while our problem might be related, it's somewhat different as I didn't have an empty Iditem field. I was finally able to delete all these empty items in a timely manner. I realised that last night there was something else probably slowing down the process : automatic updates taking place at around midnight. However, I did deactivate all pertinent column equations to make sure the process would be as speedy as possible. The question remains though... Why all these empty items ?

Happened again this morning. 386 "blank" items got created in a new grid (just created this morning). Just before this happened, here's what I did in this newly created grid (with a semi-complex source (field1 AND field2), no filter/sort criteria, and grid isn't on auto-refresh mode) : - Create a sort filter (4 fields -- e.g. : xmo, noe, links, order) - Left a trailing comma after order (sorry... Did it again...) - (don't refresh yet -- grid isn't on auto-refresh mode) - Apply "Filter applies to sub items" - F5 IQ froze. When I reopened the grid after restarting IQ : 386 items with an empty item field were there...

MANTIS

1106

Thousands of auto-generated blank items (filter or auto-assign related problem ?)

Bug

average

new

2011-02-13 14:42

Summarized the current problem as :

"Sometimes, after applying a filter or refreshing a grid (manually, or automatically after a copy/paste, IQ freezes and 100s or 1000s of items (blank) are created in the background. The only way to stop the process is to forcefully shut down IQ."