InfoQube IM - Community

Submitted by Armando on 2010/11/12 01:16

There's been many posts about sorting in the last few days

The best approach to problem seems to be :

1- Automatic sorting using columns : only when sort bar is ON.

Nothing to add here. This should/would be the default sorting mode for most users

2- automatic sorting using the Sort Text box following these transparent rules

a- The Sort text box (I prefer that expression than the shorter "Sort box", as it's too close to "sort bar" -- very confusing) should be considered an advanced interface and shouldn't be compulsory at all.

b- the sort text box should be deactivated when the sort bar is turned ON to eliminate any confusion. This not a problem as both sorting cannot really coexist (well they can and will to a certain point... ) and even if they did, it wouldn't be that advantageous.

3- manual sorting following... Oops, the question remains open...

co-existence of manual and automated sorting has always been a problem. E.g. : Users destroy their careful custom sorting when accidently clicking on a column... and then panic. More often then not, it's recoverable, but since there's no clue about that, they make the situation worse by trying to resort manually their 100s of items... :)

Now, what to do so that both modes can coexist peacefully?

A- Manual sorting is only possible when sort bar is OFF. It would still be possible to access automated sorting by using the grid context menu : sort ascending descending. This seems a nice compromise to me...

When auto-sort would be turned off, the sort text box would then be reactivated (if it was activated before). If the sort text box was also deactivated (not used : no sort criteria), then user can manually sort. This would be in most cases, since most non-advanced users wouldn't use the sort text box anyway.

B- a button to save snapshots of current manual sortings ? These snapshots would be saved in a list. Users could go back in time. This is risky and complicated.

C- Leave as is but show a popup explaining the situation when user is putting its manual sorting in danger...

"C" solution is somewhat the most flexible but.... Not sure it's very elegant.

"B" leaves room for human errors ("Oh no, I forgot to save my sorting!"). And I'm not sure about its implementation. It's not that elegant either...

"A" leaves less room but it's safer, and if you want to include some automation to your manual sorting, you can use the context menu "sort subs ascending/descending) by clicking inside the various displayed columns in the grid. Seems to me like the safest and best solution. Not too obscure and relatively intuitive.

The sortbar button name could be changed for "Auto-sort" or something like that. Something that speaks about the feature and the difference between manual and automated sorting.

When hovering over the button, a tooltip could say : "ON : Sort according to column headers. No manual repositioning of items is possible. OFF : come back to manual sorting; can't sort using column headers"

Opinions please.

Comments

Good idea Armando (discussion) - but Pierre has plans for sorting for 0.9.27 - do you know what they are?

You say eslewhere (http://www.sqlnotes.net/drupal5/index.php?q=node/1946#comment-9109):

> 1- IQ allows manual sorting. To protect that, sorting using columns (which has the potential to destroy your manual sorting) is made more deliberate if you need to activate the sort bar.

Manual sorting has to somehow become easier - but it should not infringe on 'normal' sorting in my opinion. Normal sorting should be intuitive - as like other apps as possible.

- I think that sorting by column should work out of the box without the sortbar showing.

I dont personally use manual sorting or the sortbar so I'd have to think some about your suggestions and maybe try out the sortbar a bit. (I normally only use the sortbox or I sort sub-items by a particular field.

Manual sorting is mainly used for text based outlines, where items represent paragraphs

[quote=Pierre_Admin]

Manual sorting is mainly used for text based outlines, where items represent paragraphs

[/quote]

Ah yes,

very important then to many people...

PS what I meant to say in my post above was:

Remembering Manual sorting has to somehow become easier

Personally I think that changes in the sort bar should update the sort text box (which I like to call the source sort box), but understand the problem is that the underlying cpde actually sorts at two levels - the grid (sort bar) and at the source (sort text bar).

Given that limitation I would think what you describe is the best approach with the following tweak. I would say we should be a little more explicit with the user about what is going on and point out that one is source-level and one is grid-level. So I would have a menu option that allows the user to choose between:

grid-level sort (sort bar)

source-level sort (sort text box aka source sort box)

When in grid-level sort, the sort text box is disabled. The group by column is also disabled. This is the default mode.

When in source-level sort the sort bar is disabled and column sorting by clicking is diabled.

I'll add that I think if you are in grid-level sort mode, you should be able to be column sort even when the sort bar is not visible. I don't see why it needs to be visible to do basic sorting. Outlook and Excel don't require you to open a sort bar to do basic column sorting.

So I would see the menu as the following:

sort

show sort-bar

group items by sort column

enable source sorting

The last one is disabled by default, and when selected disables the first two menu options, column sorting, grouping, and the sort bar.

Is there any actual difference to the end-user whether a sort is done at 'grid-level' or at 'source-level' ?

I mean is there any reason for me to choose one over the other ?

(question not related to *how* they are implemented - unless that has directly to do with the level of the sort)

Main drawbacks of each:

grid level: can only sort on fields displayed in the grid
source level: (1) sorts on the actual cell content, so <A>, <B>, <IMG> tags etc disrupts the sorting order and (2) currently, the UI is... minimal

HTH !

[quote=Pierre_Admin]

Main drawbacks of each:

grid level: can only sort on fields displayed in the grid
source level: (1) sorts on the actual cell content, so <A>, <B>, <IMG> tags etc disrupts the sorting order and (2) currently, the UI is... minimal

HTH !

[/quote]

if this limitation - "<A>, <B>, <IMG> tags etc disrupts the sorting order" - could be overcome, I see no reason to keep the two different 'automatic' sorts.

I'm still of the opinion that they make the whole sorting thing unduly complex and confusing.

My ideal (re column sorting):

You could have a sortbar that would apply sort based on column headers dragged into it (as is), but, on the right side of the bar (possibly as a drop-down box), you would have a box that would reflect the fields in the bar - but where you could also add fields by typing them in. This would all be based on the 'source level' sorting.

Or, how about a sortbar that you could drag fields from the properties pane into

But if the above limitation cannot be overcome I guess you have to keep both types of sorting... (I still get caught out by that occasionally, as I mainly use the sortbox)

[quote=Tom]

My ideal (re column sorting):

You could have a sortbar that would apply sort based on column headers dragged into it (as is), but, on the right side of the [sort] bar (possibly as a drop-down box), you would have a box that would reflect the fields in the bar - but where you could also add fields by typing them in. This would all be based on the 'source level' sorting.

Or, how about a sortbar that you could drag fields from the properties pane into

[/quote]

(my emphasis)

I like both ideas. That way you'd have what the sort text box allows (sorting by fields not present in the grid) and the nice UI the sort bar provides.

If one could drag fields from the properties pane, the sort text box becomes less important, although I find it a bit more reliable than the column sorting... Will have to do some more tests.

One thing we didn't mention yet : sorting history. The sort text box allows that, and it'S fantastic. I use it all the time to switch from one complex sort to the other, without having to retype everything.

[quote=Armando]

[quote=Tom]

My ideal (re column sorting):

Or, how about a sortbar that you could drag fields from the properties pane into

[/quote]

(my emphasis)

I like both ideas. That way you'd have what the sort text box allows (sorting by fields not present in the grid) and the nice UI the sort bar provides.

If one could drag fields from the properties pane, the sort text box becomes less important, although I find it a bit more reliable than the column sorting... Will have to do some more tests.

[/quote]

I really like all of this a lot. And hope I have implied as much in the other thread. I personally love using the sort box, and the only reason I touch the sort bar is to use grouping. I like grouping a lot, but the fact that I can't hack it quickly like the sort box or keep a history is really keeping me from using the sort bar at all.

What I really want is grouping in the sort box. So you could have something like this:

BatchArea ASC GROUP, BatchProj DESC

And in a perfect world, the sort bar and sort box being in sync with each other like Tom describes. So the sort box gets updated as you move things around in the sort bar (and the reverse).

Pierre, I know they are different implementations, but it seems to me it should still be possible for you to update the sort box with changes in the sort bar and the reverse. Perhaps the right implementation approach is for the sort box to not be connected to SQL at all and really just be a text representation of the sort bar. I don't see a huge need to sort on columns not in the grid anyway, if I need them I can add them to the grid. Note in this case the sort box is still around, if you take away I (and I think a few others) will scream :).

(For clarity : starting another "thread" in this discussion, summarizing my thoughts in reactions to other posts...)

From : http://www.sqlnotes.net/drupal5/index.php?q=node/2097#comment-9133

[quote=reesd]
(...) I would have a menu option that allows the user to choose between:
grid-level sort (sort bar)
source-level sort (sort text box aka source sort box)

When in grid-level sort, the sort text box is disabled. The group by column is also disabled. This is the default mode.
When in source-level sort the sort bar is disabled and column sorting by clicking is diabled.
(...)
So I would see the menu as the following:
sort
show sort-bar
group items by sort column
enable source sorting

The last one is disabled by default, and when selected disables the first two menu options, column sorting, grouping, and the sort bar.
[/quote]

I think that this is in line with "my" propositions. Some thoughts :

A "Sort menu" to switch between types of sorting :
- Users could rapidly switch between 2 or 3 modes.
- switches are all in the same spot.
Addition :
- The third mode would be manual sorting (reesd doesn't mention that possibility which I'll discuss after the second quote)

reesd, you don't mention manual sorting at all. And the reason why the sort bar has been set as it is now, as I've explained in yesterdays thread, is exactly because of manual sorting and its related dangers.

reesd :

[quote=reesd]
I'll add that I think if you are in grid-level sort mode, you should be able to be column sort even when the sort bar is not visible. I don't see why it needs to be visible to do basic sorting.
[/quote]

If you reread my post(s), you'll see that I'm suggesting that it'd still be possible to sort using the grid context menu. this would still allow some automated sorting in manual mode or any other mode mode (like source "sort text box" mode)

manual sorting problem basics : doesn't matter if the sort bar is visible or not 1) BUT we need some clear and intuitive separation between manual and automated mode, 2) AND/ OR we need some mechanism to be able to preserve/come back to manual sorting against all odds. (see note 2)

Coming back to the menu with sorting options mentioned at the beginning :

the third option could be "manual sorting mode". When this is selected, 1) a pop up could offer to keep the present sorting as a basis for the following manual sorting, 2) or discard it and go back to precious stored manual sorting.

I put what I consider important in yellow. Everything else is skipable... :)

NOTES :
(1)
[quote=reesd]
Outlook and Excel don't require you to open a sort bar to do basic column sorting.
[/quote]

In one way Excell is a good example as you need to consciously apply any kind of automated sorting. Sorting is normally manual and you need to apply the automated sorting afterwards. IQ's manual sorting mode should be like that --> Automated sorting should be deliberate and very conscious.

That said, I don't know any one who (e.g.) writes novels in Excell and who, like an IQ user, is afraid to loose his manual custom sorting by clicking on a header. Also, my version of excel doesn't allow me to click on a header to sort -- well, unless I write some macro to do it... I'm on 2003.

As for Outlook, AFAIK I can't manually sort anything, not even notes. So Outlook is not a good example : it lacks an important feature that IQ has. Plus, there's no "source level" sorting.]

(2)
The compulsory presence of the sort bar for column header sorting was one tentative to preserve the manual sorting. No sort bar --> no automated sorting, no "Oops! I clicked on the header and destroyed my novel!" moment (even though this wasn't usually completely true....). I explained the history of the whole sorting thing in the other thread and briefly mentioned some aspects here.

One question though :

Where should the sort mode menu be ? This is fundamental. No one should ever hunt for it while wondering why on earth columns headers won't sort anything when clicked (manual mode). The menu button should be "In your face".

[quote=Armando]

One question though :

[/quote]

Very much agreed, it must be "in your face"

If I start a clean install of IQ, open a sample db, and try and sort by column, it doesnt work. Now my first impulse is to click on the arrow on the right side of the column header. But that's filter. Ideally that's where I'd put it, beside the filter drop-down arrow, with a tooltip telling us what it is.

I think people who dont use manual tend to forget about it when planning ;-)

[quote=Armando]

manual sorting problem basics : doesn't matter if the sort bar is visible or not 1) BUT we need some clear and intuitive separation between manual and automated mode, 2) AND/ OR we need some mechanism to be able to preserve/come back to manual sorting against all odds. (see note 2)

Coming back to the menu with sorting options mentioned at the beginning :

[/quote]

All sounds good !

Possible Sort Menu options/summary:

Sortbar (sorting by column/s in grid)
Manual Sorting (column sort still possible via context menu)
Sortbox (sorting by *any* field/s)

[quote=Armando]

From : http://www.sqlnotes.net/drupal5/index.php?q=node/2097#comment-9133
reesd, you don't mention manual sorting at all. And the reason why the sort bar has been set as it is now, as I've explained in yesterdays thread, is exactly because of manual sorting and its related dangers.
[quote=reesd]
I'll add that I think if you are in grid-level sort mode, you should be able to be column sort even when the sort bar is not visible. I don't see why it needs to be visible to do basic sorting.
[/quote]

If you reread my post(s), you'll see that I'm suggesting that it'd still be possible to sort using the grid context menu. this would still allow some automated sorting in manual mode or any other mode mode (like source "sort text box" mode)

manual sorting problem basics : doesn't matter if the sort bar is visible or not 1) BUT we need some clear and intuitive separation between manual and automated mode, 2) AND/ OR we need some mechanism to be able to preserve/come back to manual sorting against all odds. (see note 2)

Coming back to the menu with sorting options mentioned at the beginning :

[quote=reesd]
Outlook and Excel don't require you to open a sort bar to do basic column sorting.
[/quote]

In one way Excell is a good example as you need to consciously apply any kind of automated sorting. Sorting is normally manual and you need to apply the automated sorting afterwards. IQ's manual sorting mode should be like that --> Automated sorting should be deliberate and very conscious.

That said, I don't know any one who (e.g.) writes novels in Excell and who, like an IQ user, is afraid to loose his manual custom sorting by clicking on a header. Also, my version of excel doesn't allow me to click on a header to sort -- well, unless I write some macro to do it... I'm on 2003.

As for Outlook, AFAIK I can't manually sort anything, not even notes. So Outlook is not a good example : it lacks an important feature that IQ has. Plus, there's no "source level" sorting.]

(2)
The compulsory presence of the sort bar for column header sorting was one tentative to preserve the manual sorting. No sort bar --> no automated sorting, no "Oops! I clicked on the header and destroyed my novel!" moment (even though this wasn't usually completely true....). I explained the history of the whole sorting thing in the other thread and briefly mentioned some aspects here.

[/quote]

Your right, I didn't bring up manual sorting because I don't use it that much :). I am much more interested in getting what I call automated sorting working right. In that case I think Outlook is a gold standard, and the sort bar is a step towards that.

Obviously you do use it, and I agree its important not to lose it. I think we agree there should be two modes and the user needs to explictly choose which one he wants:

Manual Sorting

Items are sorted manually through a menu choice (context or in top menu). They can be sorted just at the current level or perhaps the entire hierarchy.

Sort options (sort bar, sort box, column clicks) are disabled and/or hidden.

In this mode items are always shown in their sibling position. Running the menu item essentially changes their sibling position.

This is how Ecco and Bonsai work. And its how Excel works by default (unless you turn on filtering/tables which makes it work like sort bar).

Automatic Sorting

Items are resorted every time you do a grid refresh based on sort rules (in sort bar/box). Grouping lives here also.

Sort options (sort bar, sort box, column clicks) are enabled and visible. Optimally what Tom/you/I describe above with the sort box and sort bar working together.

Automatic Sorting does not change an item's sibling position. And sibling position is not used for sort, or maybe its used last.

This is how Outlook, web tables like Gmail, and DB tools work.

Note that if "sort-criteria applies to sub-items" is off, then you are only Automatic Sorting at the TLI level and the children are Manual Sorted.

Thanks for the follow up!

I guess the best we can do at the moment is to look for a corresponding Mantis issue and link the thread.

--------------------------------------------------------------------------------

Windows XP Home Edition, Service pack 2

Dell Vostro 1500, Ram:3gb, CPU:Core2Duo T7500 2.2ghz

Thanks to all for this interesting discussion, which I'll no doubt consider when ready to work on this again, in v0.9.27...

For reference, here is a summary of how Ecco approached this:

Ecco did not support any kind of automatic sort (Ecco-Ext does support it through something called "persistent sorts")
Ecco manual sort (i.e. up/down left-right positioning) was very easy, simply select items, grab bullets and move).
Left-click on a column did not sort, in fact, it did nothing, except start drag-drop
Right-click on the header showed header specific context menu (i.e. no sort)
Right-click on a cell allowed 3 sort-related choices
- "Sort up / down" would sort TLI only, based on the value in the clicked cell column (this excluded the item column, which in Ecco, is not a column per se)
- "Sort ..." allowed sorting:
  - of TLI only, using any grid column, including the item.
  - If the selected items have sub-items, sorting sub-items only, was also possible, for those items (and recursively)
    
    (actually, right-click was not as flexible as Tools>>Sort... due to limitations of the Ecco columns, which only supported single cell selection)
The Tickler was the only view that allowed context-parent free display, hence allowing somewhat more flexible sorting. In Ecco, context parents were often in the way when it came to sorting...
Manual positioning of sub-items were the same in all notepads (update one, all update), however, TLI position in a grid was grid-specific (a notepad contains a linked-list of folders each of which contains a linked list of TLI, all saved with the grid (aka, the notepad)

HTH !

Are there anything in the Ecco that you'd like to "borrow"?

The sort items dialog reminds me of Excels Sort dialog. It's not bad... Maybe such a dialog could be used as an interface to help fill the source bar "sort text box".

Columns filtering doesn't really need anything else than the ~~source~~ [sorry!] sort bar (IMO). Simple is better in that case.

Ecco seems somewhat less powerful in terms of sorting -- except for the manual sorting of TLIs.

I think that if we can just clarify the UI for the different sort methods in IQ, it should be as functional as Ecco, and more.

It's interesting to see the different directions people would be coming from.

Because I use IQ partially like a file manager I always thought that a normal column sort would be expected - but I see from this thread, that this is not the case.

It would certainly be nice though to (continue to) make sorting options as flexible as possible to cater to different uses of IQ

Those using IQ as a text outliner (such as Ecco, Omni-outliner), Jan is such a user, but there are many other, definitely don' t want items (i.e. paragraphs) moved around too easily (i.e. resorted on a single header click). Maintaining text order is crucial !!!

How about if the concept of a document-type grid...

Let's face it, you don't use Excel to write an essay and you don't use Word to manage tasks and your budget...

But both can be handled quite easily in IQ. To be effective in these two tasks, the UI has to adapt.

In a document-type grid, sorting would never be automatic but an action a user can do. Exactly as in Ecco, keeping the position of an item in an outline would be critical.

Comments ?

[quote=Pierre_Admin]

How about if the concept of a document-type grid...

Let's face it, you don't use Excel to write an essay and you don't use Word to manage tasks and your budget...

But both can be handled quite easily in IQ. To be effective in these two tasks, the UI has to adapt.

In a document-type grid, sorting would never be automatic but an action a user can do. Exactly as in Ecco, keeping the position of an item in an outline would be critical.

[/quote]

sounds like a good idea to me

How would you see the non-document type grid then ?

[quote=Pierre_Admin]

How about if the concept of a document-type grid...

Let's face it, you don't use Excel to write an essay and you don't use Word to manage tasks and your budget...

But both can be handled quite easily in IQ. To be effective in these two tasks, the UI has to adapt.

In a document-type grid, sorting would never be automatic but an action a user can do. Exactly as in Ecco, keeping the position of an item in an outline would be critical.

Comments ?

[/quote]

I'm one of those who uses IQ a lot to write texts (plays actually) etc.

It could be a good idea, but I must say that one thing that I like in IQ is that grids are "agnostic" so to speak.

Even when I work on something where some arbitrary (manual) order is important, I still need all the other features a normal grid has : columns, equations, etc. Otherwise, I'd just use MS Word... Word is perfect for most writing tasks. However, it doesn't have fields and equations*, etc. and so I don't use it anymore.

The only things that bothers me at the moment when I work in IQ on an a structured text are :

- Proofing

- Formating

- The fact that grid cells don't scroll well.

- Outline styles

- [The fact that when I make a mistake in a cell, in editing mode, I can't undo, and <Esc> could unrecoverably wipe a way several lines of text.]

That's it. So if the "document" grid does that AND keeps columns, equations etc., sure. And that would need to come with more sophisticated printing options [The reason I mention that is that several users here (SuperBoyAC, Jan, Jon...) seem to miss that -- personally , I don't really print much]

However, if not, a button to change sorting mode (as mentioned above) seems like the simplest solution.

* Equations : when I say equations, I don't necessarily mean stuff like Field = OtherField * 2 or SUM(children)... But lots of other functions where portions of texts are concatenated, conditionally formatted, parsed, etc.

[quote=Pierre_Admin]

Let's face it, you don't use Excel to write an essay and you don't use Word to manage tasks and your budget...

[/quote]

Just wanted to say something about that specific sentence.

You might not want to write your novel in Excel, but you might want all entries there to stay exactly where you manually put them, even if it's not an essay, but some spreadsheet/table/whatever.

It's an important fact, and I could give many example of when that would be the case.

That said, I'm very curious about the "document type" grid.

This thread is starting to be somewhat cluttered and ideas will be lost. Eventually, some kind of summary would be good.