Using topic names as indexes

Problem

Our foswiki system is growing as we slowly develop new features. After using TWiki and then Foswiki for quite a long time I realized: I know nada about Wiki Applications (hereafter WA). To get much more of the power of Foswiki we need to use forms. And when there gets more and more forms and they contain info related to each other then one starts to see some challenges... I recon DBCache(Contrib/Plugin) and VIEW_TEMPLATE as mandatory from now on for us, although it principally the same with SEARCH but I believe it's not as scalable, yet. Why oh why isn't there more for-dummies-like-me docs about that... well, here's maybe a little bit:

If you know a little bit about databases and tables you think of an index field when inserting and relating information. You can do that with Foswiki also. Just add new topic with the AUTOINC feature and have a form with formfields to hold the info. That together with VIEW_TEMPLATE (and I really recommend AutoViewTemplatePlugin until it gets into core/default plugins - look it up Foswiki:Extensions/AutoViewTemplatePlugin and Foswiki:Extensions/DBCacheContrib / Plugin ).

A topic "aTopic" with a form "aForm" is viewed with a VIEW_TEMPLATE, there's a search to bForm where b.aField is matcing (=~) aTopic's topic name, but there can also be other names in bForm.aField because it is of type select+multi (so bForm.aField = aTopic1, aTopic2, aTopic5, aTopic8). We use select+multi+values for some formfields so we can see nice choices to select from and then the topic name is what actually is saved (or several names, +multi). This, I believe, is using topic names as indexes.

Next we want to see some other info related from another form, called cForm. So bForm has a formfield "cField" which refers to topics that has the cForm. bForm.cField is also select+multi+values and has saved topic names (bForm.cField = cTopic3, cTopic5, etc).

So the challenge was from viewing a topic with aForm and gathering info from bForm and cForm topics so that the aTopic viewing page had much nice and useful info ready at hand. We needed to search all topcis with bForm.aField =~ BestPracticeTip9 ( the rendered page we look at is the topic which is index). Then from the bTopic results we want to lookup bTopic's bForm.cField and find one or more "cForm" topic names where we actually want to display the cForm.Name field. And we even want to make it clickable so onavigation is easy. I think I gave up like seven times or more but like the goof I am, I had to try a bit more. A solution I believe I found, read on.

Context

So we document our IT systems with Foswiki, not too heavy but it's ours and we have control (and it's free - both meanings). We have forms to document apps, server, routines, etc. We also use PublishPlugin and cron to generate Nagios config so when we document things correct, we have monitoring correct and linking. Now raise a hand....(tbdf)

The current challenge for me was when looking through a VIEW_TEMPLATE a an application. We want to see all routines (procedures, call it whatever) regarding this app (how to install, configure, upgrade, handle users, etc) and also which servers it running on, what it's related to, depends on, etc.

An app can be registered in several routines and be installed on several servers. We devided that an application MUST have a routine describing how it got to belong on a server, even if it would just be to refer to a standard document somewhere. That is the connection between an application and where it is installed. It got to be mapped somewhere and we think this i a good way to explain "why/how come this app is on this server?"

Solution

The following search in the VIEW_TEMPLATE for applications seem to solve the challenge. Yet no corner case found that disrupts it. I've simplified the names and removed unneeded params like web="x".

I have all on one line: 
%FORMATLIST{"$percntDBQUERY{\"ThatForm.aField =~ '%BASETOPIC%'\" 
format=\"$formfield(topics)\"}$percnt" format="[[$1][$percntFORMFIELD{\"Name\" topic=\"$1\" web=\"aWeb\"}$percnt]] " }%

Known Uses

I think it's really hard for a non dev/techie to learn to use this stuff without more easy guides. The same with layout. I hope that we can make small web wizards or templates to help with getting started using these powers. Because I believe it is often after realizing these powers that Foswiki becomes a real strong alternative when businesses are looking for capable software to do stuff.

Known Limitations

Just remember the macro expansion rule: inside-out-left-right. And repeat it a hundred times. Well, just ask in the Foswiki:Support Web, IRC or maillist if you have a gut feeling that you should get on with this.

See Also

Foswiki:Extensions.FilterPlugin which now is a part of default plugins. I also tried Foswiki:Extensions.ForEachPlugin but gave up soon.