Item10813: Change in Query search breaks applications that refer directly to the form name

On release branch you can write:

%SEARCH{
type="query"
"BlogPostForm"
}%

This no longer works. You now have to write:

%SEARCH{
type="query"
"form.name='BlogPostForm'"
}%

Meaning that the query search on trunk searches META data only if you use meta search syntax.

I bet searching on form name directly is in use a lot because it is so convenient.

-- ArthurClemens - 31 May 2011

Oh dear. This is undocumented behaviour.

Sven had to tighten up some corners of QuerySearch. As far as I know, all changes have remained within the basic spec documented at QuerySearch.

The reason for these changes is that this syntax is non-deterministic: we can't anticipate what "BlogPostForm" means until the BruteForce algo scans an individual topic. So we're doomed if we ever want the query to be executed inside a database - we can no longer do any perl that will magically make "BlogPostForm" mean a formfield on one topic while meaning a form name on another. For efficiency we need to decide what it is (field name or form name) before we throw the query at the DB.

I suppose it's not completely impossible to implement some sort of "auto-vivication" at the database (perform the expensive computation about what "BlogPostForm" means on a per-topic basis and cache that somehow), but it's already very very hard to write a hoister that compiles an efficient mapping to MongoDB, and we even have the capability of generating arbitrary javascript programs that the DB can execute - I can only imagine how frustrating this might be for an SQL hoister (though I guess you can do stored procs etc. but then that reduces portability to different SQL products).

So, the path forward:

• Consider that this behaviour was not documented (at least, not in QuerySearch)
• There were no unit tests for it (or was there? Sven?)
• Do we document this in the upgrade notes - that QuerySearch users will need to review their queries to see if they're relying on the old behaviour?
  • Is it possible to write a tool to scan for problematic type="query" searches? I guess it won't catch dynamically built queries...
• Or do we ask people writing hoisters to try even harder (it's already been a massive challenge to get imperfect coverage, even with the luxury of generating arbitrary js code to mimic perl/querysearch idioms?)

-- PaulHarvey - 31 May 2011

This is something that needed to be excised from the query language.

its only convenient to search for "BlogPostForm" if no-one makes a form field called by the same name - at that point, the query language becomes non-deterministic - which is a very bad idea.

from the task that removed this Item10520 : the docco explicitly denied the existence of this convenience.

-- SvenDowideit - 31 May 2011

I must say that it comes as a surprise to me that a query search for a simple name can mean anything else than searching for a field.

I would not recommend that it can mean both either. I consider the current 1.1 behavior a bug.

-- KennethLavrsen - 01 Jun 2011

I can see why the change is needed.

To see what impact we may face I have done a scan of trunk with all extensions. Found so far:

• BlogAddOn - fixed yesterday
• ExtensionsRepositoryContrib (ConfigureImageGallery, FastReport)
• WebsiteGalleryContrib (WebsitesList)

This is a small and manageable list - the use is not so widespread as I thought.

-- ArthurClemens - 01 Jun 2011

Still, I think it's a good point that this is useful information for the upgrader, so we should use this task to the upgrade/release notes.

-- PaulHarvey - 01 Jun 2011

And don't forget the DarkPan...

-- OlivierRaginel - 03 Jun 2011

Yes, we need to remove non-deterministic behaviour (a.k.a "shortcuts") wherever possible. Wish I'd stuck to my guns when we were designing the bloody language, now

-- CrawfordCurrie - 04 Jun 2011

I have added a note to the (new) ReleaseNotes01x02 topic.

-- ArthurClemens - 19 Jun 2011

Commenting is disabled while not logged in