InfoQube IM - Community

Submitted by WayneK on 2016/05/06 21:18

As I've been sinking more and more time into putting my research into InfoQube, I've been increasingly having thoughts about the need for some kind of export and back up to software completely independent of InfoQube. I wonder if I'm being too paranoid and am just wasting my time looking into this but I've already sunk several hours into it.

I'm looking for some method that will at least preserve the majority of my labor in a format that's readily accessible and reliable and will remain so for the foreseeable future. I don't expect anything to go wrong with InfoQube at all but I think, for me, it would provide a little extra measure of comfort if I knew that my work cannot be lost (at least not completely).

I've read the manual, looked at past threads, and tried out several possibilities. It doesn't look like there's a simple solution to this. My inclination is to go with Ecco Pro or Excel. In the case of Ecco Pro, however, there doesn't appear to be a way to preserve any fields, so I'd just be backing up the text research itself. With Excel, I can export the whole grid, preserving the field structure. But afterwards, Excel isn't as powerful or flexible as Ecco Pro for working with the text.

I could fill in more details but I'll wait to see if anyone else is interested in this topic. I might be the only one.

Wayne

Comments

I'm not clear on what you are after, or what getting it into Ecco or Excel accomplishes? Why not just implement a robust multi-version backup plan and regularly back up all the associated IQ folders?

Why does anyone need to export? There are significant sections of the Help file and forum threads on exporting, so I presume other people think it's worth something. Here are some of my thoughts (but again, I'm not sure I'm right in thinking this way).

1) Avoiding the small but (possibly) real risk that 10 years from now I have 5,000 hours of work tied up in a program that has been abandoned and no longer runs in Windows.

2) Sharing my database with people who don't want to learn InfoQube? Having my data in a program that is simpler, easier to access, has a longer

history of support, and is easier to use by others.

3) Playing with the data using a different system of tagging / filtering with the possibility of coming up with new ways to analyze the data. Sometimes when I'm working on ideas, I'll move into a new format just to get a different look (eg start working on paper, or move lists into a mind map).

4) Having a back up system in case InfoQube has trouble handling the large amount of data I'm planning to enter.. I'm already running into long wait times opening grids. Maybe the text can be accessed faster in Ecco Pro, which can be handy if I just want to quickly look something up.

Anyway, if no one seem any value in these thoughts that's fine. I was just thinking maybe someone else is going through the same process and would want to share experiences with different export options.

Wayne

[quote=WayneK]

I'm already running into long wait times opening grids.

[/quote]

That's probably because you're displaying 1000s of items at once OR because your source bar filters are very complex (mine are, but I don't wait more than a few seconds).

Usually, to avoid too many items appearing all at once, I split data in chunks using filters (date, alphabetical, or whatever -- I can give you examples if you want).

My DB isn't a Ferrari anymore, but it's still pretty responsive. Here's what I've got in my DB:

171025 items

670 Fields

941910 Text values

184102 Number values

85930 Date values

1220006 Yes/No values

251 Grids

42 Forms

A few questions (if you want -- you don't have to answer, of course) :

- How many items to you have in your DB?

- Are there situations where you really need to "see" more than 1000 of items all at once ?

- how many seconds do you wait for your "slowest" grid to load? (and what is its filter, and how many items does it display?)

IQ geek

Windows 8.1

CPU: Intel i5 2.6ghz

As mentioned below, I'm keeping most of items collapsed under TLI's. My grids generally only have 50-100 items displayed. When a TLI is expanded, that can jump to 2,000+-. That's probably the most I see at one time currently.

I'm not using filters yet. I'm still just getting the data in put and haven't tried to do much with it.

I just opened my file. The active grid has 1,700 items displayed. It took about 20-25 seconds for the grid to display, during which I got a "not responding" message. I went to another gird. It has about 3,000 items open. It took about 20 seconds display, with the same "not responding" message. I don't consider that too bad. I think the longer delays I was having earlier were a system slowdown problem. I rebooted and I think it's back to manageable waits.

I have about 5,000 total items. I think Pierre said he was opening grids displaying 20,000 items with no problem.

So it's not a big deal right now but I think I'm probably less than 5% to my total projected size (based purely on a wild guess).

Wayne

Win 7, 64 bit Cyberpower Model 1, C Series (2011) 16 GB RAM

27" Samsung S27A350H, 24" ASUS VW242H 1920x1080

[quote=WayneK]

I'm not using filters yet. I'm still just getting the data in put and haven't tried to do much with it.

I just opened my file. The active grid has 1,700 items displayed. It took about 20-25 seconds for the grid to display, during which I got a "not responding" message.

[/quote]

20 seconds is a very long time... I suspect that all items and sub-items are meeting the grid criteria and IQ is busy re-arranging items. 2 things to try:

Hide context parents and set the display mode to flat. How many items are displayed ?
What is the grid source ? Try setting only a value for the grid source field for TLI items only. An easy way to do so is:
1. Create a new grid, call it TestGrid. A Y/N field TestGrid will be created for you.
2. In your main grid, hide context parents and set the display mode to normal.
3. Press Ctrl + 1 to close all items.
4. Select all level 1 items
5. In the Properties Pane, set the TestGrid field
6. Go back to your TestGrid grid and press F5 to refresh
7. Report back the typical load time

HTH !

Pierre_Admin

IQ Designer

Pierre,

I don't want you to think I'm uninterested or don't appreciate the help, but I've been a bit tied up today. I will get back to this and follow through with your suggestions and post the results.

Wayne

Right now, I've got no reason to export my data to anything as IQ does what I want/need...

Nevertheless , it's true that one might need to move data somewhere else, for various reasons*. And the idea that data is/could be "trapped" somewhere -- wherever it is -- IS annoying (e.g. if one is doing research and data needs to be exported/imported elsewhere for calculations, comparisons, analysis, ..., because some software has better/different/more fitting functionality, etc.).

Fortunately, IQ's data isn't stored in some obscure format only Pierre knows; IQ's actual DB format, the same as MS Access, should be/is completely readable from Microsoft Access. From there it could be exported in some other simple formats (CSV) or more complex DB format (like mySQL, SQLite, etc. presumably...haven,t really researched the subject)

Note that it's also possible to work with IQ's data from other applications using an ODBC file/link. See 9. Example-Finances (a bit dated, but that,s the idea), or other discussions about ODBC fun (Rendering IQ Data to Java Cell Phone -- demo attached ).

With Pierre's help, user Mark Gregory did a lot of interesting stuff with data "linking" between DB, etc. In particular with Zotero. See Synchronisation to an external SQLite database.

Honestly, I've never really played with linking IQ's DB to other DB for anything serious. BUT, maybe other users (like Mark) who worked hard with connecting IQ to other software (Zotero) could chime in... I'd certainly be really interested. : )

IQ geek

Windows 8.1

CPU: Intel i5 2.6ghz

Thanks, Armando. I did know that about the database format and realize the chances of real, total loss are very slim.

Another reason I've started using another backup program is to just provide a check on my work and not lose anything. I know everyone has had the experience of items not being where you think they should be. As the database gets larger and larger it becomes increasingly easy to miss the fact that items have disappeared from the grids. I know, of course, that they're not really gone, but they're gone from your mind if they're not in the grid view.

To deal with this I've started keeping simple lists of what should be in each grid. This doesn't involve any kind of export but does require some extra steps just to have a separate tracker on where things are and where they should be.

Wayne

[quote=WayneK]

[/quote]

IQ doesn't mess with the users' items, but users sometimes forget some important settings (autoassigns, equations, inheritance, multiple parents, source/filter settings, etc.) and items get lost.

As with everything else, anything can be "lost" and one needs to religiously follow precise processes to make sure it doesn't happen.

There are strategies one can use to check for "lost" items once in a while. I do that about once every 2 weeks : I use specific filters built around specific combinations of fields and operators to help me retrieve those forgotten items.

I also religiously tag every single item I create (the process is partly automated, and I use popups), and I review all created items every day (with the right filters it's actually pretty fast). Yes. Sounds a bit excessive, but it's the best way I've found to not loose anything.

[Edit : what can also help -- depending on the situation -- is to work with items under a parent (i.e. instead of working with items as TLIs in a grid, you have a few TLIs and other items are under). Items don't "disappear" as easily from under parents, unless you move them, filter them out or erase their parent links; but if you've filtered them out, you can always use the "show all sub items" option to see what's really there.]

I know Pierre doesn't really agree with that but to me also, orphan items are/can be a problem. Definitely. If you can't see something, even if it's there somewhere, it's basically lost (temporarily or even for ever if you completely forgot about it and will never think of retrieving it...).

I think that there should be algorithms to help users bring "lost items" to the surface. IIRC, Pierre wrote a couple filters a while ago to achieve just that, but I can,t find them at the moment. (like I said, I already use my own strategies.)

IQ geek

Windows 8.1

CPU: Intel i5 2.6ghz

Nice tips.

I'm still in the process of inputting data and gradually figuring out how I want to organize it. I've found myself gravitating towards keeping things grouped under TLI's just as you suggest. I know there were some pretty good discussions of the pro's / con's of the two approaches in the old Ecco Pro newsletter.

This actually touches on something I'd been thinking about starting a thread on, but maybe you can suggest something here. When you're putting the bulk of your items underneath TLI's, what's the simplest way to "free them up" and allow the subitems to be sorted independently of the TLI's? I think I want to be able to do that but haven't really gotten into that portion yet.

I was thinking of assigning all the subitems to a separate grid, then turning off context parents so only the subitems are displayed. Then they can all be sorted together without intereference by the TLI's (I think). Does that sound like a good approach?

Wayne

Win 7, 64 bit Cyberpower Model 1, C Series (2011) 16 GB RAM

27" Samsung S27A350H, 24" ASUS VW242H 1920x1080

[quote=WayneK]

[/quote]

Hierarchical and heterarchical (or flat) organization serve different purposes so I use both "grouping under TLIs/complex hierarchical structures", and "flat lists" extensively.

So how would one "free" subitems? Simple : define something about those items you need to "free" from their hierarchical structure (i,e, define something so that they can be displayed as autonomous items, as TLIs, in other grids). Define what differentiates those items from other items (either data that's already specifically different or data you'll specifically add/remove).

Here's a short (simple and insignificant) example:

1- Create a new sample file

2- open the AdrsBook Grid. There you see contacts with phone calls under them and other stuff.

BUT, lets say you'd like to see all your past phone calls in a flat list (maybe to be able to later order them by date, or categorize then in some other way -- whatever)

3- so create a grid named "callDate"

By default, its source will be callDate, which is the same as a field that already exists.

4- open the grid if it isn't already open. Set the source & filter like that (to view the source & filter bar, one way is to click on Grid (menu)->Source bar):

source = callDate

filter = item like "call*"

Make sure to Activate the filter (click on the filter icon beside the filter text box, turn it ON -- it can be applied only to TLIs or TLIS + Subitems)!

Here you go: you've freed all your "Calls" from the contacts. In that grid, you'd also be able to directly create items that will have a callDate assigned to them (by default, this field will assign today's date) sort them as a flat list, etc.**

You could also add a few more columns to your grid, so that your items can be displayed in a different context, with a different "meaning".

----

Another thing you could qu icky try, is to use date/text/boolean filters to only show a subset of your sub-items.

For example, in the AdrsBook grid,

1- select the "Outline (normal)" view mode

2- expand all items to the second level (Ctrl+2) and refresh (F5)

3- open the source bar and insert that filter:

| calldate > #2008-06-05#

(the vertical bar "|" separates the filter that applies to TLI, and the part that applies to subitems. In the filter above, I'm just filtering subs, so there's nothing before he "|")

(Again : make sure to Activate the filter : click on the filter icon beside the filter text box in the source bar, and turn it ON) .

Refresh the grid (F5) : you should see only the sub items with a call date that's > to 2008-06-05.

(If the grid allows for the automatically assigned dates to be erased when items are moved, then items will /might (depending on the source) lose their date if you demote them -- If you don't want that, you can change that in the grid settings.)

There are so many ways to use IQ's display flexibility , that it would be too long to give a detailed presentation... but that should give you some ideas.

IQ geek

Windows 8.1

CPU: Intel i5 2.6ghz

Armando,

Thank you very much for taking the time to write out a detailed explanation. That's exactly the kind of guidance I needed to move forward with more confidence. Your generous help on this forum (along with, of course, Pierre and the other regular posters), is so valuable to those of us still learning to use this powerful program.

Wayne

[quote=WayneK]

Armando,

Wayne

[/quote]

You're welcome. Hopefully, my explanations aren't too obscure... Don't hesitate to ask questions while you're trying out stuff.

IQ geek

Windows 8.1

CPU: Intel i5 2.6ghz

Hi Wayne!

I understand your concern. As others have mentioned, IQ uses the same database engine as MS Access. I'll add a "get my data back" command which will export a series of excel files

HTH !

Pierre_Admin

IQ Designer

Thanks, Pierre. There's a simple export to Excel right now, but I could only get it to export the TLI's. To get the subitems, I've had to expand all the levels, then manually select them and copy/paste to Excel. It's worked well in my limited tries but obviously isn't as handy as having a single command that would export all levels without having to expand and manually select them. Maybe that's what you have in mind.

Also, if you have time, I'd appreciate your advice on freeing up subitems as mentioned in my post above.

Wayne