Submitted by gregory on 2015/04/30 11:20
<EDIT> I've added the source of a couple of functions to make the source code more complete, and clarified Figure 1. </EDIT>
 
 
However, I can find no definitive documentation on what the acceptable syntax for equations is. Things that you might expect to work do not do so. For example, you cannot combine strings using an ampersand character & as you can in VBScript.
As a consequence, I wasted quite a lot of time over the last couple of days until I realised that to combine strings I would need to write a VBScript function.
 
The following screenshot shows what does NOT work. Moreover, it fails silently – the only way to find out that it hasn't worked is to try it and see that it has had no impact on the field cc. It still doesn't work if you explicitly recalculate the field.
The reason, no doubt, is that this syntax is not acceptable in this context: cc = aa&bb. Field cc is highlighted at point 1 in figure 1; the equation is highlighted at point 2.
 
Figure 1: the equation which derives field cc

In the end, I have written VBScript functions:
 
Function combineSubstrings(Separator, firstString, secondString)
' This function assumes that the desired result string consists of two substrings separated by
' a separator substring.
' Mark GREGORY

     combineSubstrings = ftrim(firstString) & separator & ftrim(secondString)
End Function

Function reverseSubstrings(Separator, search_in)
' This function assumes that a string consists of two substrings separated by
' a separator substring. It reverses the two principal elements and returns
' a new string consisting of the second element then the separator substring
' then the first element.
' Mark GREGORY

Dim firstElem, secondElem
     reverseSubstrings = after(separator, search_in, "first") & separator & before(separator, search_in, "first")
End Function
 
These two functions themselves depend upon further functions:
 
Function ftrim(inString)
ftrim = LTrim(RTrim(inString))
End Function

Function includes(look_for, search_in)
'This function returns false if the string is not found at all.
' Mark GREGORY
includes = InStr(1, search_in, look_for)
End Function

Function findlast(look_for, search_in)
'This function returns the start position of the last occurrence of a looked-for string
'in a searched-in string. -1 is returned if the string is not found at all.
' Mark GREGORY
Dim strlen, lastfound, nextfound
strlen = Len(search_in)
If strlen < 1 Then
    findlast = -1
Else
    lastfound = InStr(1, search_in, look_for)
    If lastfound = 0 Then
        findlast = -1
    Else
        nextfound = InStr(lastfound + Len(look_for), search_in, look_for)
        Do While nextfound > 0
            lastfound = nextfound
            nextfound = InStr(lastfound + Len(look_for), search_in, look_for)
        Loop
        findlast = lastfound
    End If
End If

End Function
 
Function before(look_for, search_in, first_or_last)
'This function returns the part of the searched_in string before an occurrence of a looked_for string.
'By default, this is the first occurrence of the string; an optional parameter can be set to indicate that
'the last occurrence is sought - the default is to take the first occurrence.

Dim strlen, found, last_occ
If LCase(first_or_last) = "first" Then
    last_occ = False
Else
    If LCase(first_or_last) = "last" Then
        last_occ = True
    Else
        last_occ = False
    End If
End If

strlen = Len(search_in)
found = InStr(search_in, look_for)
If found <= 0 Then
    before = search_in
Else
    If last_occ Then
        before = Left(search_in, findlast(look_for, search_in) - 1)
    Else
        before = Left(search_in, found - 1)
    End If
End If
End Function

Function after(look_for, search_in, first_or_last)
'This function returns the part of the searched_in string after an occurrence of a looked_for string.
'By default, this is the first occurrence of the string; an optional parameter can be set to indicate that
'the last occurrence is sought - the default is to take the first occurrence.
' Mark GREGORY

Dim strlen, found, last_occ
If LCase(first_or_last) = "last" Then
    last_occ = True
Else
    last_occ = False
End If


strlen = Len(search_in)
found = InStr(search_in, look_for)
If found <= 0 Then
    after = ""
Else
    If last_occ Then
        after = Right(search_in, strlen - findlast(look_for, search_in) - Len(look_for) + 1)
    Else
        after = Right(search_in, strlen - found - Len(look_for) + 1)
    End If
End If
End Function
 
I make use of these functions in, for example, a tagged classification scheme which I am currently working on.
 
The field TagKind uses combineSubstrings - see point 1:
 
Figure 2: the equation which derives the field TagKind
 
The field KindTag reverses the order so that kindVal comes before Tagvalthis field uses the function reverseSubstrings:
 
Figure 3: the equation which derives the field KindTag
 
We can use these fields in order to implement the many to many relationship between kind and tag. To create a new tag | kind combination, we create a new item and select from the drop-down fields as follows:
 
In the next screenshot, I have already selected a tag value - CITSA 2009 - and I am about to select a kind value:
 
Figure 4: choosing a KindTag

In the next screenshot, figure 5, I show the results after I have selected that kind value and after the field equations have fired; note that I have also reordered the data by ascending KindTag. Point 1 shows the TagKind; point 2 shows the corresponding KindTag.

Figure 5: The fields have now been correctly updated
 
Getting these examples to work has taken about two working days. I hope that by sharing these examples, others will be able to save some of that time.
 
WHAT I AM SUGGESTING NEEDS DOING:
 
  1. I am hopeful that InfoQube has used a component for which documentation is available showing the acceptable syntax and semantics of the various kinds of equations. If not, then some sort of documentation should be written from scratch. Again, I am willing to help but will need input from Pierre.
     
  2. Additionally or alternatively, it would be great if InfoQube produced some sort of error message or helpful diagnostics when incorrect syntax or fieldnames are used in equations, rather than simply failing silently, as it does at the moment.
 

Comments

seems to have to do with the way equations are parsed for translation into VB. I could be wrong. Yes, there are a few quirks (using field names -- like "item" in functions strings in IQ UI can wreck a function) Hopefully, they will be ironed out. 
Thanks!
 
-------------------------------------------------------
Windows 8.1
Sony Vaio S Series 13 (SVS131E21L)
Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

gregory

2015/05/03 07:59

In reply to by Armando

Hi Armando.

Your comments are very interesting. I do not really understand how InfoQube actually works behind the scenes. I do know that field names are actually the names of queries on the underlying relational database. Clearly as a field is about to be used, its properties are taken into account and these may include equations. The parsing process, the rules which govern it and how these are applied are not clear to me.

I don't quite follow what you are saying about restrictions on field names such as item. Have you documented this somewhere?

Best regards
Mark
 
Mark GREGORY, Redon, France - GMT +1/+2; EST +6

Armando

2015/05/04 00:42

In reply to by gregory

[quote=gregory]
Hi Armando.

Your comments are very interesting. I do not really understand how InfoQube actually works behind the scenes. I do know that field names are actually the names of queries on the underlying relational database. Clearly as a field is about to be used, its properties are taken into account and these may include equations. The parsing process, the rules which govern it and how these are applied are not clear to me.

I don't quite follow what you are saying about restrictions on field names such as item. Have you documented this somewhere?

Best regards
Mark
 
Mark GREGORY, Redon, France - GMT +1/+2; EST +6
[/quote]
 
I'm only guessing.
 
Note that cc = aa & bb actually works. Well, i think it does.  My guess is that your syntax might be faulty...? you wrote =aa&bb, but I'm not sure if it would work in VBScript without any spaces between the concatenation operator and the operands... maybe.
 
In any case I've struggled with equations too in various ways, especially problems related to mixed row/column equations 
 
& the fact hat certain strings like "item" will crash some of my functions. It'd be a bit long to fully document fully but it mostly involves:
 
1- the use the string "item" in string operations longer than 18 chars 
2- in functions' parameters involving the item field
3- destined to be returned to the item field itself
 
e.g. a function scrubbing the item field's content with a regex pattern (I'm using colors to better illustrate the points previously mentionned ) :
 
item = SearchReplaceRegExPat (item, "(.*)(<b>\[À CLASSER DANS.*\]</b>)(.*)", "$1$3") 
 
Sorry for making your eyes bleed...
 
So, basically, if the item field in this example would contain " this is the item field and it will crash the function because of it contains the item term inside it", the SearchReplaceRegExPat  function would crash, and the  returned value would be empty. AFAIK... it's been a while.
 
there are other cases (see row/column equations problem thread)
 
[slightly edited -- first draft was too... quickly drafted]
 
-------------------------------------------------------
Windows 8.1
Sony Vaio S Series 13 (SVS131E21L)
Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

gregory

2015/05/04 07:33

In reply to by Armando

Well, once again, you are entirely correct… =aa & bb does indeed work; thank you for the suggestion. This reinforces my contention that the rules governing equations should be fully documented. Note also that it is necessary
completely to recalculate the field cc from the Field Properties dialogue.
 
Figure 1 shows that field cc has not been recalculated:
 
Figure 1: Field cc is not recalculated
 
In order to create this screenshot, I created a new item, clicked under aa, added the value aa5, clicked under bb and added the value bb5.

I have then tried what I can to get the Row equation which governs the field cc to fire. Nothing I have tried works. In particular shift F9, which should always recalculate a field, is not effective.

The only way in which I can get the field populated is to go into the field properties dialogue and completely recalculate the values for cc. Now this is both inconvenient and quite likely not to be what the user wants – since all values of the field will be recalculated.
 
Figure 2 shows the results.
 
Figure 2 Field cc has now been updated

One other slight oddity, unrelated to the title of this post, is the fact that when I input a value to the field  aa, it is automatically preceded by a space. This can of course be deleted, but is an unexpected and unwelcome phenomenon. I think that it is fairly recent.

I have followed your earlier posts about difficulties with equations, particularly where both row and column equations are in play. The difficulties which you encounter should by now have been resolved; it is sad that they have not been since this and similar difficulties reduce user confidence in the product.

When you say that the example you give fails, I assume that you mean that it fails noiselessly, without an error message?
 
So there are now four problems which need attention:

1. The syntax and other rules governing equations need to be more thoroughly documented.
2. Equations should not fail silently – there should always be an error message.
3. Attention should be given to your long-standing concerns about equations, particularly those involving both rows and columns.
4. Why is a space being inserted at the start of a value?
 
Mark GREGORY, Redon, France - GMT +1/+2; EST +6

Armando

2015/05/04 12:10

In reply to by gregory

I think Pierre is aware of these quirks and he'll be in a better position than me to explain why and how this happens, and why he can,t fix it now.
 

[quote=gregory]
 
In order to create this screenshot, I created a new item, clicked under aa, added the value aa5, clicked under bb and added the value bb5.

I have then tried what I can to get the Row equation which governs the field cc to fire. Nothing I have tried works. In particular shift F9, which should always recalculate a field, is not effective.

The only way in which I can get the field populated is to go into the field properties dialogue and completely recalculate the values for cc. Now this is both inconvenient and quite likely not to be what the user wants – since all values of the field will be recalculated.
 
 
[/quote]
 
what you report is strange but I've experienced it many times. I think that "= cc & bb" should be re triggered every time  cc or bb is modified; you shouldn't have to press shift+F9 or whatever. 
 
haven't got the time to find out why it does it, but I've experience it (functions not working automatically and suddenly working).
 
This is a bug in IQ that happens when an equation is new (maybe), a grid hasn't been refreshed, etc. Can you try after having refreshed the grid, or close/reopen ?
 
 
-------------------------------------------------------
Windows 8.1
Sony Vaio S Series 13 (SVS131E21L)
Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

gregory

2015/05/04 12:47

In reply to by Armando

You're a genius (but you knew that already...). Closing all IQ bases then restarting has caused a series of equations to start working - all this after several frustrating days! Thanks!
 
Mark GREGORY, Redon, France - GMT +1/+2; EST +6

Armando

2015/05/04 14:01

In reply to by gregory

[quote=gregory]
You're a genius (but you knew that already...). Closing all IQ bases then restarting has caused a series of equations to start working - all this after several frustrating days! Thanks!
 
Mark GREGORY, Redon, France - GMT +1/+2; EST +6
[/quote]
 
There you go!
 
Yes, these equations (row/column equations weirdnesses and other) things need to be fixed.

Pierre :  these are a huge time drain and the source of much frustration when something doesn't work as expected and... data is sometimes lost. Yup. : - (  Other things in IQ are the source of much happiness. If only the somewhat  annoying aspects could be fixed.
 
 
-------------------------------------------------------
Windows 8.1
Sony Vaio S Series 13 (SVS131E21L)
Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

Hi All !
 
There are quite a few distinct issues here...
 
The first I'll address is field parsing (handles the case of "item" in a string and "bb&cc")
 
I'm proposing that fields referenced in equations be enclosed in square brackets, always.
 
so [aa]=[bb]&[cc] would be valid
but aa = bb & cc would not
 
This would be a simple and safe syntax.
 
Eventually, it would also support field naming (which currently breaks equations...)
 
OK with this change ?
 
 
 

gregory

2015/05/04 15:33

In reply to by Pierre_Admin

Yes, [bracketed field] is fine by me!
 
Mark GREGORY, Redon, France - GMT +1/+2; EST +6

Armando

2015/05/04 16:06

In reply to by Pierre_Admin

[quote=Pierre_Admin]
Hi All !
 
There are quite a few distinct issues here...
 
The first I'll address is field parsing (handles the case of "item" in a string and "bb&cc")
 
I'm proposing that fields referenced in equations be enclosed in square brackets, always.
 
so [aa]=[bb]&[cc] would be valid
but aa = bb & cc would not
 
This would be a simple and safe syntax.
 
Eventually, it would also support field naming (which currently breaks equations...)
 
OK with this change ?
 
[/quote]
 
This is fine, theoretically. But I have so many equations/fields, that it probably take me 2 days to correct all that and make sure it works. That time, I don't have; it would probably mean that I'd have to cancel a trip to Paris, and other things.
Is there a way to implement that without breaking everything, OR in a way that's easier than to go through each fields with equations and bracket field references one by one ?
 
 
-------------------------------------------------------
Windows 8.1
Sony Vaio S Series 13 (SVS131E21L)
Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

Pierre_Admin

2015/05/04 16:13

In reply to by Armando

[quote=Armando]
This is fine, theoretically. But I have so many equations/fields, that it probably take me 2 days to correct all that and make sure it works. That time, I don't have; it would probably mean that I'd have to cancel a trip to Paris, and other things.
[/quote]
 
If you've got that many equations, perhaps you should consider consulting a professional... 
 
More seriously, I'll write code to automatically convert equations... will that ease the transition ? You'll only need to scan all fields to ensure that conversion worked as expected.