InfoQube IM - Community

Submitted by Tom on 2010/08/03 05:07

Related manual page: 3. Backup your database, Encryption

Discussion re backup started in this thread: load / save layouts doesn't save horizontal grids layout

[quote=Pierre_Admin]

> well my IQ auto-backups are timestamped and tend to take up a lot of space.

True, but if you use dropbox, you can turn off timestamped backups and let it keep all changes to the backup file (.SNBKA and .SNBKM)

> Am I correct in thinking that currently nothing can copy the IQbase while it's in use?

No. The OS can copy the file, no problem. In fact, the current backup does exactly that, copy and rename the IQBase.

[/quote]

[quote=Tom]

[quote=Pierre_Admin]

> well my IQ auto-backups are timestamped and tend to take up a lot of space.

True, but if you use dropbox, you can turn off timestamped backups and let it keep all changes to the backup file (.SNBKA and .SNBKM)

[/quote]

thanks Pierre, I'll try that

[quote=Pierre_Admin]

> Am I correct in thinking that currently nothing can copy the IQbase while it's in use?

No. The OS can copy the file, no problem. In fact, the current backup does exactly that, copy and rename the IQBase.

[/quote]

in that case I can probably use my backup programme to do local backups as well

[/quote]

I wanted to go into more depth so I started a new thread, queries to follow:-

Comments

If you're just using Dropbox as backup it seems like a good idea to just keep the backup file in the Dropbox folder as Pierre suggests

1) does the folder *.sndb.files folder not get backed up by IQ ? (I believe not)

2) I currently have my main IQbase & 'files' folder residing in my Dropbox folder (A). If I copy the IQbase file and the related folder elsewhere (B) - the file in the new location (B) will read the MHT files from the old location (A) - this could get confusing (especially if a user didnt notice this)

I would then be backing up the base file from B to A and backing up the files folder from A to B. This is naturally doable but I wanted to add something to the Backup manual page, and that's too complex.

Any tips?

3) Can I (easily) get the file to read the file folder in the same directory first -

4) should this not be default behaviour (to look for the files folder in the same directory first, & only then to follow last known path)?

That's what I do. Same thing explained there, actually : http://www.sqlnotes.net/drupal5/index.php?q=node/1595#comment-6618

1- I just backup to a folder located in the dropbox folder (E:\My Dropbox\IQ bacp)

2- My IQ DB backup is set to backup once every 45 min (a smaller value is disturbing, I find, as IQ freezes during Backup).

3- I don't time stamp.

4- the "*.sndb.files" folder is backed up by my own backup software (Syncback), once a day. I don't mind that as loosing 1 or 2 MHT files would be the worst scenario.

Voilà.

As for your questions :

1) No, it doesn't... however a text only copy is kept in IQ. If I'm wrong, Pierre will correct me. ;)

2) Maybe Pierre could add an option to change the location of that folder in the options dialog. And offer to copy all files form the old location, when changed. What do you think ?

3) Not sure about that one.

4) Yes, I'd say so. See #2

> That's what I do. Same thing explained there, actually : http://www.sqlnotes.net/drupal5/index.php?q=node/1595#comment-6618

ooohhh - I missed that or else I forgot to try it out

Belated thanks for the tip !

& thanks for the feedback.

You say:

"2) Maybe Pierre could add an option to change the location of that folder in the options dialog. And offer to copy all files form the old location, when changed."

Does " And offer to copy all files form the old location, when changed" mean: if I move my IQbase, I would be asked (on opening) if I want to move the files folder as well ?

[quote=Tom]

Does "And offer to copy all files form the old location, when changed" mean: if I move my IQbase, I would be asked (on opening) if I want to move the files folder as well ?

[/quote]

What I meant is that if you enter a new path for the files folder, IQ could offer to move the content to the new location.

Now, what you propose is also interesting -- that IQ would offer to move the files folder (on opening) even when only the database is moved (.sndb ).

so Pierre, I guess the main question is:

Re moving the associated files folder:

Would it be possible to "add an option to change the location of that folder in the options dialog. And offer to copy all files from the old location, when changed."

and, if this is possible, should I add it to mantis ?

Hmmm, I guess I'm doing things all wrong or leaving myself open to problems

I save/open dB to dropbox

sndb is in dropbox

I backup to local HD w time stamps, every 60 mins, for 3 days

document type files that I've linked to are in special file on local backup HD

I compact on exit

I'm sure this can be improved upon. Suggestions.

[quote=jan_rifkinson]

Hmmm, I guess I'm doing things all wrong or leaving myself open to problems

I save/open dB to dropbox

sndb is in dropbox

I backup to local HD w time stamps, every 60 mins, for 3 days

document type files that I've linked to are in special file on local backup HD

I compact on exit

I'm sure this can be improved upon. Suggestions.

[/quote]

This looks good to me. But it depends on how safe you want to be.

- If you have your IQ DB outside dropbox and backup (without time stamp) to drop box instead, this means you won't have to close your IQ DB for dropbox to sync it.

- Otherwise, it's always good to back regularly to an external storage device.

I summarized my plan a while ago on DonationCoder.com here. it hasn't changed much, except that I also use dropbox for my IQbase... That's because IQ is one of the most used app on my system now (in that order exactly : IQ, Firefox, Outlook, Titlelog, NotePad++, find & Run Robot + Everything, MediaMonkey), and, of course,I can't afford loosing a lot of data all at once.

[quote=Armando][snip]

- If you have your IQ DB outside dropbox and backup (without time stamp) to drop box instead, this means you won't have to close your IQ DB for dropbox to sync it.

- Otherwise, it's always good to back regularly to an external storage device.[/snip]

[/quote]

Armando, let me be sure I'm clear on this.

Backing up w/o a time stamp creates a file that I can open on another machine w/o closing my IQ dB.

Let's assume I do that & I change something on the backup, i.e. on another machine

Won't my IQ dB become out of sync?

I think I'm not understanding you well.

[quote=jan_rifkinson]

Armando, let me be sure I'm clear on this.

Backing up w/o a time stamp creates a file that I can open on another machine w/o closing my IQ dB.

Let's assume I do that & I change something on the backup, i.e. on another machine

Won't my IQ dB become out of sync?

I think I'm not understanding you well.

[/quote]

Ok, I think you're talking about syncing between different computers while were talking about backing up.

We use Dropbox purely for backup here, in this example. If you want to use dropbox to sync an IQ base between different machines, sure, you absolutely need to have your DB in the dropbox folder -- that and your IQ files folder. Of course, you could also use the system mdescribed here and also back up to the drop box folder and dropbox will create versions for your different backups so no need to use the timestamp feature in IQ. But it all depends on how much space you have in your DropBox account, etc.

[quote=jan_rifkinson]

Hmmm, I guess I'm doing things all wrong or leaving myself open to problems

I save/open dB to dropbox

sndb is in dropbox

I backup to local HD w time stamps, every 60 mins, for 3 days

document type files that I've linked to are in special file on local backup HD

I compact on exit [/quote]

not wrong or problematical !

It depends on two things -

how often you close your IQ file (cause it only gets backed up when closed) &
how often you want to back it up online

if #2 is greater than #1 then you might consider changing it to the way Pierre & Armando recommend.

I havent changed it yet - I'm waiting to see what Pierre says about the related files folder - i.e. I'm still using pretty much the same setup as you (file is in dropbox, make local backup)

HI all !

I'm a bit confused here. I have the IQBase :

in a dropbox folder
auto-backup every 15 min
Everything else is off

That way Dropbox:

Syncs the .SNDBA file (no need to close the IQBase to get protection)
Syncs the .files folder too

So both the IQBase and the .files files (HTML content) is fully backed up. Also, it is possible (though not exactly quick) to view / restore both of these to previous version if need be (i.e. item deletions, HTML content deletions / loss)

Seems pretty fool-proof to me

[quote=Pierre_Admin]

HI all !

I'm a bit confused here. I have the IQBase :

in a dropbox folder
auto-backup every 15 min
Everything else is off

That way Dropbox:

Syncs the .SNDBA file (no need to close the IQBase to get protection)
Syncs the .files folder too

Seems pretty fool-proof to me

[/quote]

What's confusing ? The difference here is only that you put everything in your DropBox folder while I only put the backups...

I wasn't sure it would be good for me to do that as as my DB + files take a lot of space. And I don't like the idea of having all that in the /my dropbox/ path. However, when other collaboration features will arise, I might change my mind.

Question : If I move my IQbase + *.files folder to the dropbox folder, will the transition be smooth?

>Question : If I move my IQbase + *.files folder to the dropbox folder, will the transition be smooth?

A move (or a copy + rename) is handled quite well as IQ will reconnect to the files in the .files folder when it does not see it in the previous location

(A straight copy is more problematic, as the files in the previous .files folder are still available)

[quote=Pierre_Admin]

>Question : If I move my IQbase + *.files folder to the dropbox folder, will the transition be smooth?

A move (or a copy + rename) is handled quite well as IQ will reconnect to the files in the .files folder when it does not see it in the previous location[/quote]

copy + rename, sometimes I completely forget the obvious solution(s)

[quote=Pierre_Admin]I have the IQBase :

in a dropbox folder
auto-backup every 15 min
Everything else is off

That way Dropbox:

Syncs the .SNDBA file (no need to close the IQBase to get protection)
Syncs the .files folder too

Somehow I was thinking I had to have the file separate from the backup (it's ingrained behaviour at this stage)

But this looks good.

Ideally cloud backup is in addition to local backup (or backup to somewhere physical) so best to use a backup programme to create another backup somewhere (I know you all prob know that, I just saying it for the record)

Forgive me but I'd like to start over again.

Backup vs Syncing are two different functions.

First scenario is regular backup of ALL IQ data which at this moment I think I have scattered so......

Let's talk specifics

What is the NAME of the IQ dB, i.e. where data is kept

What is the NAME of the file/directory where HTML bits are kept

What is the NAME of the file/director where linked files are kept (mine is called "linked files" and is a separate folder)

I have found that I have some "linked files" in two same named folders in different directories so I have to clean this up.

What's the best way to accomplish that?

When I make auto-backup every x minutes, specifically what file NAMES am I backing up

When I make time stamped auto-backups every x minutes, specifically what file NAMES am I backing up

I am REALLY confused now.....

Summary : IQ backups (from the options) backup your DB. That's it.

> What is the NAME of the IQ dB, i.e. where data is kept

You named it yourself, most probably, so it's hard to say... When you open IQ, it should probably show the name in the titlebar.

Something.SNDB

> What is the NAME of the file/directory where HTML bits are kept

*\Something.SNDB.Files\

>What is the NAME of the file/director where linked files are kept (mine is called "linked files" and is a separate folder)

None. You decide. I personally :

1- create a link to a file using Explorer : alt-drag-drop to the *\Something.SNDB.Files\ (to facilitate that I have a link to

*\Something.SNDB.Files\ on my desktop and dragging something to it will drag it in the folder.)

2- Then I link this link (drag drop) to my IQ base

> I have found that I have some "linked files" in two same named folders in different directories so I have to clean this up.

>What's the best way to accomplish that?

Find out which link are used and which ones aren't and deleted them. You can see which links are used in the properties pane, going from items to items. Or you could

1- go in the search grid and enter a filter like "Filepath is not null" in the filter text box

2- and make the Filepath visible in the grid (drag drop from properties pane... ) so that you can easily see all items at once.

Modify as needed.

> When I make auto-backup every x minutes, specifically what file NAMES am I backing up

You,re backing up your IQ base only. Not the *\Something.SNDB.Files\ .

Note however that IQ contains the text version of everything that's in your *\Something.SNDB.Files\ folder

>When I make time stamped auto-backups every x minutes, specifically what file NAMES am I backing up

Same as above.

Really, it's not that complicated, I think.

Also see : 3. Backup your database, Encryption

Armando, you say:

> Note however that IQ contains the text version of everything that's in your *\Something.SNDB.Files\ folder

I added this to the manual backup page:

> Text copies of the HTML Pane content are stored in the database SNDB file

Is that correct ? Does that mean it stores text copies of emails that have been saved as well (these are also saved in that folder - well, when saved from Thunderbird using the extension)

Yes, thanks Tom. It seems correct. Everything that's saved in the HTML pane (through the universal clipper, withing the main IQ interface, etc.) is saved in the DB as text only (so : no image, etc.).

However, AFAIK, if you open and view a pdf file in the HTML pane, and maybe a .doc, the text isn't saved in the DB.

so there are only two files to worry about. Currently I'm worrying about 3

the sndb.files folder can / should contain all the pdf, doc, xls, etc files that I have linked to in IQ?

When one of these is backed up w/o time stamp, both are backed up? do I have that right?

currently I have them in a folder called "Linked Files" so I guess I have to figure out how to move them over

OK, let's rewind a little. Information viewed in IQ can be from 4 sources:

YourIQBase.sndb: Grids, Calendar, embedded HTML content
YourIQBase.sndb.files folder: Embedded HTML and captured content (universal clipper, browser extensions) saved as MHT or as an HTML file
A link to a file anywhere on your computer (or a link to an item in a file, such as an Outlook item link)
A URL from the web

To backup these:

Item 1 : IQ has options to perform backups, timed or not
Items 1 and 2: Copy by some means. Putting these in a Dropbox folder is a "fool-proof" and automatic method. In bonus, you get revisions
Item 3: HD backup software
Item 4: well, you can't, but Internet Archives can help
Items 3 and 4: If the linked file is HTML in nature (email, HTML, MHT), the text content is saved in the IQBase (see item 1 above)
Items 2, 3, 4: If Options>>This file>>Data Storage >> Save a copy of the MHT in the database is checked (not recommended), MHT and HTML files will also be saved in the IQBase. If you later on want to view that item and the rich text file not available, the file will be re-created, using this saved "embedded" copy.

HTH !

[quote=Pierre_Admin]# Items 2, 3, 4: If Options>>This file>>Data Storage >> Save a copy of the MHT in the database is checked (not recommended), MHT and HTML files will also be saved in the IQBase. If you later on want to view that item and the rich text file not available, the file will be re-created, using this saved "embedded" copy.[/quote](my emphasis)

Pierre, I added that info to the backup page:

can you clarify why embedding MHT etc info in the database is NOT recommended - is it simply a space issue ?

also dont understand [EDIT/ I see you mean when in browse mode, the pages you view are not saved]:

#4 "A URL from the web" - ~~do you mean stuff in the HTML pane - images - that only show when you're online?~~

~~or do you mean that saving a URL is not the same as saving the webpage ?~~

>can you clarify why embedding MHT etc info in the database is NOT recommended - is it simply a space issue ?

Yes it is mostly a space issue. That and also that IQ is not pretty smart at finding those files, if / when they've been moved

Just to make sure :

[quote=jan_rifkinson]

the sndb.files folder can / should contain all the pdf, doc, xls, etc files that I have linked to in IQ?

[/quote]

No. Anything you link to your DB stays where it is. It's a link, not an importation.

[quote=jan_rifkinson]

so there are only two files to worry about. Currently I'm worrying about 3

the sndb.files folder can / should contain all the pdf, doc, xls, etc files that I have linked to in IQ?

When one of these is backed up w/o time stamp, both are backed up? do I have that right?

currently I have them in a folder called "Linked Files" so I guess I have to figure out how to move them over

[/quote]

as said by the other's linked files are just that - backing them up is a separate job as such.

Not sure what you mean by the last line there - do you want to move them over to the same directory as your IQ database?

If you do want to move them and they were all in one place and they all going to the same destination, it's relatively easy:

select all linked items - the FileRef field is always ticked for file links so you could select them using that

change the FilePath to the new path e.g. F:\IQ\LinkedFiles\ - click OK to apply to all of them

That's it

Could you have a look at the backup page Jan and let us know what you think of it ?

I found it getting kind of complex but I also find it hard to avoid all that info - just see if it makes sense!

~~Backup - Backing up IQbase~~

[edit by admin] 3. Backup your database, Encryption [/edit]

[quote=Tom][snip]

Could you have a look at the backup page Jan and let us know what you think of it ?

I found it getting kind of complex but I also find it hard to avoid all that info - just see if it makes sense!

Backup - Backing up IQbase[/quote]

Tom when I clicked on the link it brought me right back to this page but, yes, I'm the perfect guinea pig to see if it makes sense to me & I'm glad to do it.

[quote=jan_rifkinson]

[quote=Tom][snip]

Could you have a look at the backup page Jan and let us know what you think of it ?

I found it getting kind of complex but I also find it hard to avoid all that info - just see if it makes sense!

Backup - Backing up IQbase[/quote]

Tom when I clicked on the link it brought me right back to this page but, yes, I'm the perfect guinea pig to see if it makes sense to me & I'm glad to do it.

[/quote]

Oups :-)

I added the wrong node number, unable to edit the last post so I'll add link here for anyone else looking:

3. Backup your database, Encryption

Jan gave me some advice for the writing up of the manual Backup page so I've been trying to simplify it

Would be grateful if anyone who has time could read through it to see if okay

I realised I dont understand this one:

# 2.2.4 - Limit the number of backups created:
This option deletes all but the last timestamped auto-backup of each day that are at least n days old.
Delete is performed once per day, on the first occasion after midnight.

"This option deletes all but the last timestamped auto-backup of each day that are at least n days old"

so if I have that set to 10, it sounds like it will keep one backup per day older than 10 days old - is that correct

I always presumed it just deleted everything older than 10 days old but think I misunderstood it?

[quote=Tom]

Would be grateful if anyone who has time could read through it to see if okay

[/quote]

It looks and sounds wonderful!

[quote=Tom]

# 2.2.4 - Limit the number of backups created:
"This option deletes all but the last timestamped auto-backup of each day that are at least n days old"

so if I have that set to 10, it sounds like it will keep one backup per day older than 10 days old - is that correct

[/quote]

I believe that is correct. I turned the option on, so I can verify after midnight!

Is it okay to extend this thread with another question/case?

I'm in the position of wanting both to sync and backup my DB in Dropbox, so that I can work on it on my desktop at home and my netbook on the road (my license says this is cool, as long as I don't have Infoqube opened at the same time on both). So I would like to have the .SNDB and .SNBKA in my Dropbox folder. This would work fine, except that my Dropbox has a different path on each computer: basically it's in (C:) on my desktop and (D:) on my netbook. An issue arises because there is no way apart from the path to specify where to put the BKA (for example, an option to put the BKA in the same folder as the DB). Does this mean I have no choice but to send my BKA outside of Dropbox to some local destination whose path is equivalent on both computers? (edit: Or, of course, move my Dropbox folder, which only disrupts a kind of elegance of my file structure).

You could use one of the 2 following methods:

Create a junction so that c:\dropbox is the same as d:\dropbox
Create a drive (with SUBST), say drive G: on each machine which points to the proper dropbox folder

Of the two, option 2 is probably the simplest, but this mapping must be done on each reboot, as part of the autoexec or startup folder.

(why not move the dropbox folder so it has the same path ? I suppose you have a good reason for this...)