Feature Proposal: Add a rule-based strategy to AutoViewTemplatePlugin to derive an edit/view template

Motivation

Provide a means to apply a view template based on the topic name

Description and Documentation

For now the AutoViewTemplatePlugin implements two modes of operations - section and exist - that either of which can be chosen from. A third strategy might be a rule-based one that derives the view template based on a set of rules that a topic name is checked against. This way a view template can be applied to a topic even though it does not contain a form or a section for it. This is configured using a hash array in configure and is applied to all of the site.

Examples

$Foswiki::cfg{Plugins}{AutoViewTemplatePlugin}{ViewTemplateRules} = {
  'WebTopicList' => 'WebTopicListView',
  'WebSearch.*' => 'KinoSearchView',
  'Tasks\.Item.*' => 'Tasks.ItemView',
};

This will apply the WebTopicListView to the WebTopicList topic in all webs, the KinoSearchView to all WebSearch and WebSearchAdvanced topics in all webs and the Tasks.ItemView to all Item topics in the Tasks web.

In addition the fixed way of specifying the plugin mode to either exist or section is replaced with a priority list where all three strategies can be combined, e.g.

$Foswiki::cfg{Plugins}{AutoViewTemplatePlugin}{Mode} = 'rules, exist';

This will first try to find a matching rule and then try the normal exist strategy by analyzing the form name of the current topic.

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

AutoViewTemplatePlugin.zip

VOTE

A vote is in progress for this proposal. Vote is open to anyone with an interest in Foswiki. Feature proposal votes do not require Association membership. Please vote with a clear Yes or No. You can vote "Neutral" to signal interest in the process but it will not change the result. Continue the debate below if you like. You can change your vote until the deadline. Do not add other people's votes based on statements from IRC etc. People must edit this topic themselves.

Votes starts on 08 Dec 2009 and ends 15 Dec 2009 0:00 UTC (at midnight between Monday and Tuesday).

Name	Vote Yes/No/Neutral	Comment
MichaelDaum	Yes		edit
KennethLavrsen	No	reasons given below	edit
MartinSeibert	Yes		edit
CrawfordCurrie	No	reasons given in discussion, below	edit
MarkusUeberall	No	reasons given below: make/keep it a non-core plugin first	edit
AndreLichtsteiner	Yes	backwards compatibility	edit
SvenDowideit	No	reasons given in discussion, below	edit
MichaelTempest	No	Essentially the same reasons given by Sven and Crawford	edit
PaulHarvey	No	reasons given below. Make AutoViewTemplatePlugin non-core.	edit
RafaelAlvarez	No	I still don't see a use case that cannot be implemented in a simpler way (see below). I would not mind a friendly fork of the plugin, but please consider the alternative I'm proposing.	edit
OlivierRaginel	No	As Sven suggested, merging the current functionnality into the core, and keeping advanced ones like what MD wants into this plugin makes much more sense to me than anything else. I'd love to hear from OliverKrueger too, as he coded the first part.	edit

-- Contributors: MichaelDaum - 03 Nov 2009

List of Concerns (Work in progress by Sven)

1. Where is the 'Why foswiki needs this statement (or use cases) (Raf and Sven)
   • If this is just a way to create upgrade sensitive replacements for the shipped webs, the its much more user friendly to create a new web, set SYSTEMWEB to that new web and INCLUDE the topics you don't want to change..
   • if thats not what you have in mind, it'd help to state your purpose&use case
   • The motivation to automate assigning a view templates and edit templates to a topic remain the original ones when the AutoViewTemplatePlugin was originally created. I.e. they are not able to replace a web. They allow to assign a view to a model speaking in MVC language. Even individual hand-assigned view and edit templates have a place in Foswiki for ages. I assume that people are aware of the possible use cases for these in general. So I won't elaborate on that. The only thing that is new in this proposal is to extend the process view templates are assigned automatically using the AutoViewTemplatePlugin by deriving it from the topic name. This has got a couple of advantages as has been discussed already. See AddAutoViewTemplatePlugintoFoswiki and UpgradeSafeCustomisationsOfSystemTopics but also How to improve your WebIndex in Foswiki. My personal main motivation to have rulebased view templates is to blend in SolrPlugin's version of WebSearch and WebSearchAdvanced automatically in every web of a Foswiki, i.e. without touching any of these topics assigning a view template manually. -- MD
2. by having the settings in configure, it means only the few users who have access to configure can use them (AndrewJones, SvenDowideit)
   • This has already been discussed below in full length. Conclusion was: first, this is a real expert setting that should not be put into the hands of a normal user. It potentially can mess up all of the site: '.*' => 'Sandbox.EvilViewTemplate' Second: it would be very geeky and clumsy to cramp a rule set into a linear preference variable. This easily gets out of hands even for moderate rule sets. In essence, this setting is best kept and edited within configure -- MD
3. 'WebSearch.*' => 'KinoSearchView', doesn't really help
   • Don't understand "doesn't really help". May I assume that the original motivation for (1) view templates and edit templates are understood (2) that automation of view and edit templates is understood (3) that extending the automation process as outlined here is understood? -- MD
4. The VIEW_TEMPLATE and SKIN path system is already too complex, and this adds another level of surprise to the user
   • Is this a general critic on AddAutoViewTemplatePlugintoFoswiki or specific for RulebasedViewTemplates? -- MD
5. lack of unit tests for the existing system means the impact of this change can't be examined
   • I will only continue coding if the concept of rulebased view templates as proposed here has been accepted -- MD
6. i am pretty sure there is a better way to do what you want to acheive, but because you've not told anyone the use case, we can't find the best solution
   • Please let me know if you have any idea of an alternative approach! -- MD
7. recalling the big side effects the we had when the skinpath work was done last time, and when VIEW_TEMPLATE was first added, caution is very necessary
   • Of course -- MD
8. This change is to apply VIEW_TEMPLATE (and presumable EDIT_TEMPLATE) to topics that do not have a DataForm. Which implies that its a way to secretly mix-in output into a topic
   • which sounds interesting, but has theside effect of preventing our users from knowing what is happening - view?raw=on on such a topic will result in mure confusion.
   • This is a critic for view templates in general and does not relate to rulebased view templates. And I share this concern: view templates taking effect during a raw=on can look strange sometimes, even for view templates assigned manually. Not sure what I can do about it within the scope of this feature proposal -- MD

Discussion

See also the discussions at AddAutoViewTemplatePlugintoFoswiki and UpgradeSafeCustomisationsOfSystemTopics.

-- MichaelDaum - 03 Nov 2009

This is a very nice enhancement.

The only problem is that by having the settings in configure, it means only the few users who have access to configure can use them. Other users won't be able to use them in their own webs, neither can users in the AdminGroup set them up without gaining access to configure.

-- AndrewJones - 04 Nov 2009

Well cramping the rules into a preference variable would come with a significant performance penalty parsing it out again on each click. I'd expect those rules to change very rarely maybe only on deployment of a new extension or wiki application. That's why I think it is safest to keep them in configure.

-- MichaelDaum - 04 Nov 2009

Ok, that's fair enough

-- AndrewJones - 04 Nov 2009

another 'problem' for the configure setting, is that our PERL hashs' don't help the admin select from a set of options.

oh, and I don't think that 'WebSearch.*' => 'KinoSearchView', is a good way to do this :/ as it doesn't solve the addibility of functionality - see for example what Crawford and I had to do to add the TagMePlugins' tab to the WebSearch - we need to find a way to enable plugging the functionality into the distributed topics... A similar example is the long existing, and hard coded TipsContrib 'widget' placed in the Main topic..

I suspect that rather than putting it into a hash in configure, we might be better off working out using the skin path - and implementing the choices and additives via tmpl/TemplateTopics..

(er, ok, so if that happens, I'll apply the template precompilation patch I had - that way we also avoid the performance penalty.

-- SvenDowideit - 05 Nov 2009

SKIN path and VIEW_TEMPLATEs are two independent dimensions. I don't see a contradiction.

With rule-based view templates the standard topics can remain as crappy as they are

From here on you'd create an independent cascade of view templates that does not touch the original topic content (upgrade safety) giving you exactly the needed flexibility.

-- MichaelDaum - 05 Nov 2009

I assume it it a mistake that committed date is missing. The proposal topics are not picked up by the application unless there is both a committed developer AND a date of commitment because we need the date for the 14-day rule. I add todays date now.

I raise concern against this.

The feature as such is fine. But being able to set the rule only in configure makes this feature only useful to the sort of installation where all users are seen as noobs with no rigths and everything related to making applications is down to a single person. Ie using Foswiki more as a CMS and not as a wiki where people are able to create powerful applications. I do not want to encourage this kind of use of Foswiki. It goes against what I personally view as the core values of Foswiki.

The idea that a view template can be set based on a naming scheme of topics is good. But normal users with no configure access should be able to use it. Ie. the rules must be possible to set in e.g WebPreferences. Otherwise its value becomes too limited. It must be possible to make this with a reasonable performance.

It is fine that the default can be defined in configure like we also do now in TablePlugin. And if an admin absolutely has to lock the feature to himself he can finalize the setting in SitePreferences.

-- KennethLavrsen - 05 Nov 2009

See arguments against preference variables above.Performance

Having both configure AND web preferences has always been perceived as a total mess by newcomers trying to configure Foswiki. The _default webpreferences is plain garbage. Having some important hash mapping in there is no good idea.

Our new configure interface is exactly the right place to handle stuff like that.

"Normal" users won't mess with things on that level anyway. These settings will always be protected by rather strict access settigns to the web preferences ... which in most cases is the same person that has got access to configure anyway.

-- MichaelDaum - 05 Nov 2009

Michael. You clearly do not understand our end users and clearly show no respect for them and their innovation and creativity.

On the performance side, let us see some numbers. Right now it is just an undocumented statement.

The ability for ANYONE using Foswiki (and TWiki) to create powerful application from the browser is essential. It is a core value of Foswiki. If we start turning our precious tool away from wikiness and towards webmaster/sysadmin syndrome then we loose everything we have built up the past 10 years.

I am deeply concerned with some of the proposals we see lately that go towards preventing access or making tailoring of Foswiki so geeky that you have to pay a professionel consultant to setup a site and create an application. I sincerely hope this is not really the case.

I want a Foswiki that all users can use and change - the wiki way. And that means clear and simple features that all can understand and access to change them for anyone.

Clearly a person in a large organisation with maybe 50 webs should be able to define a view template or edit template rule for the web he is maintaining by setting a simple WebPreferences macro. Configure contains far too many dangerous stuff to hand out the password for it to 50-100 people.

In the 1.1 and 2.0 plans we are supposed to have a database implementation for things like settings and then the performance issue is not an issue.

The configure way is clear supporting only one thing - the one webmaster syndrome. It is non-wiki.

-- KennethLavrsen - 05 Nov 2009

can we please cut the crap

-- MichaelDaum - 05 Nov 2009

Guys, I need a decision on this one. This is already implemented, documented and ready to be checked in. See attachment. I really need this feature as some of the next plugins will be deployed best by adding rules to the AutoViewTemplatePlugin's config.

-- MichaelDaum - 16 Nov 2009

given that you've done nothing to address the concerns and ideas raised by others, having some code done isn't supposed to be a way to force it. Guess someone will have to read the unit tests in the attached code to see if its really the bet way forward, rather than just the best you can think of?

my first concern is that

'WebSearch.*' => 'KinoSearchView',

doesn't address the additive-ness needed to support the search features we already have. Specifically, it does not address the need to have the existing topic search, tags search, and attachment searching. just over-riding a topic with another is not enough.

my second, is that I can't glean from the example, why I would want to have a WikiUsers and WikiGroups topic existing in every web, or how to specificy web locality, or even different templates for the same named topic in different webs.

it sorta feels like an incomplete proposal?

-- SvenDowideit - 16 Nov 2009

This proposal is not about how to write additive templates per se. However it provides means to start writing them without having to touch the topic underneath. The layers of templates constructed on top should be designed in a reusable way of course. Not the scope of this proposal.

I don't know either why you would have a WikiUsers or WikiGroups in every web. A view templates rule won't blend them in.

Different view templates per web can be specified by using the appropriate rule as illustrated in the example above. In addition web-specific and skin-specific templates can be blended in following the normal template search path.

The main concern against this added feature as far as I get it is that it is configured using configure only and does not support a way to write view template rules via preference variables. I tried to answer why I did not implement that and why I think that the trade-off between flexibility and complexity isn't worth it imho.

-- MichaelDaum - 16 Nov 2009

It would be helpful to know why this change is useful (at least I would like the advantages explicitly stated).

As for the ability to specify the mapping in a preference property, I think the best way would be to have a property that can be defined globally in SitePreferences or locally on each web or topic. The performance impact is no greater than what we have now.

The problem with that approach is that it is not easy to specify the kind of "hash" that is needed for this plugin using our current Preference mechanism.

Anyway, if this change is to ease the maintenance of the whole site, it is ok for me if the configuration is managed only in configure. Power Users can still use the other models, and the VIEW_TEMPLATE and EDIT_TEMPLATE preferences to achieve the same result.

Just for the records, I'm not raising a concern.

-- RafaelAlvarez - 16 Nov 2009

Kenneth and I have been talking on the summit about this and we agreed that it might be dangerous to give users this level of control: an accidental catch-all view template rule could affect a lot of topics in a harmful way.

-- MichaelDaum - 25 Nov 2009

KennethLavrsen, SvenDowideit, please review your concerns. Don't let this stale, please.

-- MichaelDaum - 01 Dec 2009

I still feel my concerns are valid, and unaddressed. I am starting to ponder another two -

1. the realisation that AutoViewTemplatePlugin actually usurps or modifies the SKIN path system (yes, i know its sort of orthogonal, but I don't think in a good way), and
2. that without unit tests for this plugin and then adding even more functionality to it feels dangerous.

I argee that we would like to have VIEW_TEMPLATE automatically set based on topic type, and that it is interesting to use this mechanism for customisations of distributed topics, but I really do wonder if expediency isn't going to lumber us with more poorly designed hard to maintain corner cases.

just to prompt my thinking from Fosiki::UI::View.pm

    my $template =
         $viewTemplate
      || $query->param('template')
      || $session->{prefs}->getPreference('VIEW_TEMPLATE')
      || 'view';

    # Always use default view template for raw=debug, raw=all and raw=on
    if ( $raw =~ /^(debug|all|on)$/ ) {
        $template = 'view';
    }

    my $tmpl = $session->templates->readTemplate($template);
    if ( !$tmpl && $template ne 'view' ) {
        $tmpl = $session->templates->readTemplate('view');
    }

-- SvenDowideit - 01 Dec 2009

Sorry, Sven. I don't understand your case. Could you be a bit more clear, please. Thanks.

Problem is I start to need this feature really badly. But now that AutoViewTemplatePlugin is protected by FollowsReleaseProcess I am somehwat stuck. What can I do to resolve this deadlock?

-- MichaelDaum - 01 Dec 2009

Here's the irc logs of a short chat with Sven. We weren't able to clarify.

[11:45] <MichaelDaum> Hi Sven. Can we have a small chat about http://foswiki.org/Development/RulebasedViewTemplates
[11:45] <SvenDowideit> in a few days yes :/
[11:45] <SvenDowideit> i need tocontinue thinking about what i'm feeling (technically)
[11:46] <MichaelDaum> I'd like to understand your concerns and address them accordingly
[11:46] --> mfladischer has joined this channel (n=fladisch@sh018.meduni-graz.at).
[11:46] <SvenDowideit> i also have a worry that you're trying to rush core stuff innapropriatly
[11:46] <SvenDowideit> but my technical concerns are multiplying the more i look at this area
[11:47] <SvenDowideit> it was already hacked to death when TW did the skin path work last time
[11:47] <MichaelDaum> can you please be a bit more concrete
[11:47] <SvenDowideit> thats the point - i can be more concerete, but taht will take more time
[11:47] <SvenDowideit> whereas if you want fast, you have to work it out more
[11:47] <MichaelDaum> give me a shorty to chew on
[11:47] <SvenDowideit> there are 4 shorties in the topic already
[11:47] <MichaelDaum> that I don't get
[11:48] <SvenDowideit> (i'm very deep into SAP atm)
[11:48] <SvenDowideit> 1) there is no unit tests for the already exisintg bit
[11:48] <SvenDowideit> that needs to change
[11:48] <MichaelDaum> okay
[11:48] <SvenDowideit> 2) you made no unit tests either
[11:48] <SvenDowideit> 3) the existing code really should not be in a plugin
[11:48] <SvenDowideit> 4) i am pretty sure there is a better way to do it
[11:49] <SvenDowideit> 4 is based on the short time i've looked at the code
[11:49] <SvenDowideit> 5) recalling the big side effects the we had when the skinpath work was done last time, and when VIWE_TEMPLATE was first added, caution is very necessary
[11:49] <MichaelDaum> I still feel I don't really get your point...besides the obvious critiques wrt unit tests and code that could very well be core than plugin 
[11:50] <SvenDowideit> yes, my opints are too vague, partially i think because you're not as familiar with the history of the related core core
[11:50] <SvenDowideit> whereas you're just looking from the POV of the simplicity of adding the function to the plugin
[11:50] --> ichbinsnuhr has joined this channel (i=54b9a663@gateway/web/freenode/x-ttamvapodhjayjgf).
[11:50] <ichbinsnuhr> hey :)
[11:50] <SvenDowideit> which imo has a good chance of eventually biting us all
[11:50] <SvenDowideit> hey back
[11:51] * SvenDowideit really has to go focus on other stuffs :(
[11:51] <MichaelDaum> would you prefer to revert the decision to make AutoViewTemplatePlugin a core plugin?
[11:51] <ChaMar> SvenDowideit: Thanks, I will try that
[11:52] <SvenDowideit> I'm notconvinced that adding a plugin is the best way to add such low level functionality to the core
[11:52] <SvenDowideit> especially without unit tests and more examination
[11:52] <SvenDowideit> i didn't look til now, because i assumed there was more thought put into it
[11:52] <SvenDowideit> (and because i'm busy elsewhere already)
[11:52] <MichaelDaum> okay so all of the AutoViewTemplatePlugin is causing you unease
[11:53] <MichaelDaum> I am okay to take it out of the core again...if that would allow me to go forward. right now I am stuck.
[11:53] <SvenDowideit> my point is that i think the functionality should be in the core
[11:53] <SvenDowideit> but taht if it is in the core, a plugin is pretty much the least appropriate place
[11:53] <SvenDowideit> because it does essentially change the skin path
[11:54] <MichaelDaum> would you accept rulebased view tempaltes if it was integrated core code?
[11:54] <SvenDowideit> um, rulebased perhaps
[11:54] <SvenDowideit> cfg'd in a hash in configure, i dunno - as i say, i think there are probly better more userable ways to cook it
[11:54] <MichaelDaum> for now I need it as a plugin as the clients are on stable branch and cant wait for 1.1
[11:55] <SvenDowideit> lack of non-trivial use cases makes it harder to think through, and thus it takes longer to think out the issues too
[11:55] <SvenDowideit> y, that thats a very bad reason to rush core functionality
[11:55] <MichaelDaum> what can I do to help to come to a conclusion?
[11:56] <SvenDowideit> work thought the concerns (yes, hard work is hard)
[11:56] <SvenDowideit> and unit tests
[11:56] <SvenDowideit> non-trivial cases
[11:56] <SvenDowideit> my next intended work on it is to re-examine the docco and Codev topics on the skin path work TW did
[11:56] <SvenDowideit> because those changes still worry me too - and this jsut ups the complexity
[11:57] <MichaelDaum> I will start on unit tests as soon as the proposal is accepted 
[11:57] <SvenDowideit> but that will take a few days of work on my side
[11:57] <SvenDowideit> shrug
[11:57] <SvenDowideit> my advise is that you make the non-core plugin, and make the unit tests now
[11:57] <MichaelDaum> a fork?
[11:57] <SvenDowideit> that way you have a chance of convincing others that this is actually the best idea
[11:57] <SvenDowideit> whereas if you keep waiting, you'll fail to convince anyone
[11:58] <MichaelDaum> Ive been talking to Kenneth on the summit about configuring rules.
[11:58] <MichaelDaum> for views
[11:59] <SvenDowideit> there's another oddity
[11:59] <SvenDowideit> i still can't grok why its limited to view&edit
[11:59] <MichaelDaum> we basically had two conclusions (a) it is hard to configure (b) it is dangerious if you get it wrong
[11:59] <SvenDowideit> but, to be 'frank'
[11:59] <SvenDowideit> its not jsut hard to configure
[11:59] <SvenDowideit> its hard to code without tests for the corner cases
[12:00] <MichaelDaum> hard from a usability pov
[12:00] <SvenDowideit> and possibly even to discuss changing the intentions of the corners
[12:00] <SvenDowideit> iirc we had to force TW to change some of the side effects several times
[12:00] <SvenDowideit> and it was a really un-fun bit of work by the end of it
[12:00] <MichaelDaum> which cornercases are there...the code is actually pretty trivial
[12:01] <SvenDowideit> the code seems pretty trivial
[12:01] <SvenDowideit> but the entire skin path thing isn't
[12:01] <SvenDowideit> view_template is only a tiny part of the whole
[12:02] <MichaelDaum> I still feel I don't get a grip on what you mean. I would not know what to do actually based on your comments.
[12:03] <SvenDowideit> which makes me worry that you don't have a full grip on the way the core does all that :(
[12:03] <SvenDowideit> which is why i have a concern, and why i feel i have to go do the research
[12:03] <MichaelDaum> okay lets end this. thanks for the short chat.
[12:04] <SvenDowideit> hope it'll help :/
[12:04] <MichaelDaum> not really
[12:05] <MichaelDaum> I will add the logs to the proposal topic

-- MichaelDaum - 02 Dec 2009

As far as I understand what Sven says, he is concerned about the AutoViewTemplatePlugin adding an extra level of complexity to the template resolution process and thus would prefer to either make it an integrated core feature instead of it being a plugin or revert the decision to distribute it with Foswiki by default all together.

I am not sure if he is directly concerned about the features proposed here (rulebased view templates) but more about the automation of view templates in general.

-- MichaelDaum - 02 Dec 2009

Current state:

• Kenneth seems to have agreed that having configuration in topic space is too dangerous / error-prone / geeky
• Sven "feels he has to go do the research" to clearly express his concerns.

Ways to resolve this:

1. Kenneth's and Sven's concerns are addressed and both remove themselves from ConcernRaisedBy. Precondition: concerns are expressed in an understandable way that allow concrete actions, or
2. A community vote decides on top or flop, or
3. AutoViewTemplatePlugin is forked.

Are the any other ways to unblock this feature proposal? Please understand that it is impossible not to decide on it. Forking AutoViewTemplatePlugin is the very last resort that I want to prevent by all means. I'd also prefer to add a deadline up to which a decision is made. Let's say 2 months are more than enough for discussing this.

-- MichaelDaum - 03 Dec 2009

mmm, I thought about your proposal to have a 2 month timelimit on a 'concern' and I strongy say no. Til now, we have treated ConcernRasiedBy as a trigger requiring the developer / suggester to overcome those concerns, not as something that can be ignored just because that suggester doesn't understand them.

If we change the process to ignoring concerns, then we're not doing our long term development any favours.

I also dissagree with your 'Current State'

• Sven feels that there are a large number of problems with this proposal, and has tried to list a large number of them, but as the proposer claims not to understand them, and has shown no progress in doing so, will, when time permits, do some extra research (that the proposer should really be doing).

I've started to fill in a List of Concerns paragraph above - and the more work I do, the less information you seem to have provided.

-- SvenDowideit - 04 Dec 2009

Thanks for compiling the List of Concerns. I will address them one by one.

-- MichaelDaum - 04 Dec 2009

I have responded to the list above .

-- MichaelDaum - 04 Dec 2009

mmm, yes, responded, but pretty much didn't address any of them - the key point here is that there is a lack of explicit details from you, preventing any of us from further analysing the root cause of your perceived need. This then means that we can't suggest any sufficiently detailed alternatives, specifically because we can't compare to your use cases. You're also evading the concerns that can be interpreted as applying to the entire AutoView idea by implying that they are thus not relevant to this feature. This is incorrect, they apply specifically to this proposal.

if your only current need is to replace WebSearch for SolrPlugin, then as I pointed out previously, this is a totally inadequate short term hack, that raises the complexity without solving the real problem.

To expand on that specific - replacing all webs' WebSearch with SolrPlugin's will still not allow the end user to have a search page that gives them access to TagMe's search, regex and query search (for prototyping and creating inline SEARCH's and Solr's. In short, magically replacing one topic with another's contents doesn't give the core what it needs, and adds more of what it doesn't.

considering your question _ if the concept of rulebased view templates as proposed here_ - made me think - your regex hash mapping is not a rulesbased system at all. And as your only use case is talking about non-DataForm topics, is a hack.

-- SvenDowideit - 04 Dec 2009

SolrPlugin will have a callback mechanism of its own which will allow external plugins to enrich documents before they are committed to the index. That's how TagMePlugin or any other meta data makes its way into the facet search.

The facet search itself is a very complex wiki application that won't fit into a standard WebSearch interface. A custom view template will allow to reimplement it on a web-by-web and application-by-application base. I can't elaborate this in full length here. What I think that I have done in sufficient length is to outline the need to override standard wiki applications such as WebSearch using a view template.

This in essence is nothing new as a VIEW_TEMPLATE variable can be set manually already. Nor does this proposal offer anything radical. Most of the innovation - automation - has been done by Oliver Krüger. All I offer here is a third strategy in addition to the existing two. Deriving the view template from the topic name instead of the form name is dead simple and straight forward with a lot of very attractive use cases.

So I hope to see you guys more supportive and appreciative on this idea.

The remaining question is not whether to implement rulebased view templates or not. The remaining questions is only: does it find its natural home in AutoViewTemplatePlugin or will our disagreement here entail a fork of this plugin?

-- MichaelDaum - 07 Dec 2009

Coming back to this proposal and looking at it again I still have concerns.

• The feature will be for the admin only as it is limited to those or maybe even the one with access to configure.
• It is very high on the NerdoMeter score. The feature is simply constructed from its very requirements as a complex feature. Imagine the text needed in configure to explain to the admin what this thing is.
• As Sven points out the feature will create an element of surprise for the end users that will not have a clue why things behave differently than what they expect from looking at the topic content. At least with the form based rule they will not see the effect on normal topics.
• Putting the feature as described here in a preference variable cures my original concern but Michael convinced me that the ease of screwing up a complete site by defining this setting wrong makes it a very bad idea making this feature available to the general user.
• On the other hand the the AutoViewTemplatePlugin as it is today is a real gem. The simple rule that you can link a view template to the use of a form is relatively easy to understand and of general use to all non-admin users. I have already have super good use of it. Extending it so you can use other rules and not requiring a form is a food idea.

Somehow this is one of those features where I feel that the best approach would be to go back to the beginning and rethink if the same functionality could be added in a simpler and safer way. I have great doubts myself what to think.

Creating a fork of the plugin may be the best short term approach. You will not hear a single work of critique from me if you do that Michael. Something like a RulesBasedViewTemplatePlugin.

Michael deserves a decision on this. Since there is concern the best and only approach in a democratic place like Foswiki is to have a vote which we announce on the discuss mailing list and give people a week to cast their vote. I will put the vote at the top to people can continue debating.

-- KennethLavrsen - 08 Dec 2009

Forgive me while I go back to basics, and consider how the template used for a script is selected, and how it might be documented in an understandable way - I find the discussions above totally obscure. At the moment we have the following mechanisms:

1. The skin
2. The cover over the skin
3. The *_TEMPLATE preferences (edit and view scripts only)
4. AutoViewTemplatePlugin section and exist

I'm never sure what order these work in, but suffice to say, it's incredibly hard to know how the template for a particular page view was arrived at. We are now talking about adding a new selector,

• AutoViewTemplatePlugin rule

If we reconsider the requirements, we want to be able to select a template - for any HTML page generated by any script - based on the following criteria (with applicable mechanisms from the list above):

1. A default (1)
2. A preference in the web the topic is in (1, 2, 3)
3. A preference in the topic (1, 2, 3)
4. The name of the topic (*)
5. The name of the web (*)
6. The name of the form attached to the topic (4)
7. The value of a form field (?)
8. URL parameters (1, 2)

So I can see that this proposal fulfills a clear need (*). (Note that I have ignored "the user displaying the topic" as a criterion; I really don't think it's useful)

Frankly the AutoViewTemplatePlugin worries me; usage is complex, and even more so with this new rule-based approach, and the documentation is complete but very hard to read. The documentation on the rule-based approach is similarly incomplete - it has taken me quite some time to fully understand it from what was written here, and I'm still not sure I fully grasp it. I'm afraid I have to vote "No". I just don't think enough thought has been put into it yet.

Aside: IMHO a much cleaner and more general approach would use a conditional set to combine the features of the query syntax with the skin=cover mechanism. For example,

• SET{"VIEW_TEMPLATE" value="ItemView" if="form.name='ItemForm'"}%
• SET{"EDIT_TEMPLATE" value="ItemEditView" if="name~'Item*'"}%

I appreciate that's not being discussed here, and the vote is on the specific mechanism.

-- CrawfordCurrie - 08 Dec 2009

Good and constructive idea, Crawford.

However, the %SET statement would become a clumsy beauty to cover all of the forms and topic names in a wiki. In addition it would need to be touched whenever someone defines a new form. Not easy to maintain. Performance decreases with the numbers of defined forms in a wiki to derive a view template from. Not so using the auto view template mechanism. It does the same in near constant time:

• form name stragegy: get form name, rewrite it to a view template name
• topic name strategy: iterate over rules in given order and check pattern, first match points to view template to be used.

-- MichaelDaum - 08 Dec 2009

I really like the proposal w.r.t. to the functionality (it surely adds a degree of freedom), but OTOH I share the concerns regarding the added complexity (both regarding code maintenance and usability). Like Kenneth, I would really like to see some kind of RulesBasedViewTemplatePlugin first in order to be able evaluate this in practice. Without a number of success stories (regarding a plugin) from third parties and potentially some modifications on the way in the first place (which would also enable both users and contributors to collect some statistics and play with it), I fear that this mechanism would always be subject of unfruitful discussions.

Therefore, w.r.t. to this vote, my answer has to be "No". (Should another proposal/vote be needed in order to introduce some kind of hooks for the forementioned plugin, I'd clearly support that--but I want it to be (at least) a two-step process.)

• Markus: this is already implemented here as part of this proposal. Have a try. -- MichaelDaum - 08 Dec 2009

-- MarkusUeberall - 08 Dec 2009

An additional rule on how a template is derived from add definitely complexity to the system. On the other hand, a rule based on the topic name seem to me easier to be grasped by the users. It's easier to communicate, understand and put actively in use by users.

Correct me if I'm wrong, but the proposal asks to add a third rule to a non-core plugin and is not breaking compatibility? So, why not?

-- AndreLichtsteiner - 08 Dec 2009

Andre, this is a core plugin - or at least is in the trunk development branch that is foswiki 1.1.

It saddens me that we're rushing to a vote on something that I think reqlly just requires more information from Michael, and at this point feel I would have to vote no for something that in a different form I also think i want.

That said, I offer alternative 3...

the code in this plugin should never have been brought into trunk as a plugin. Its a trivial piece of code that already has a relevant place - in fact pretty much in the code I pasted earlier in this topic.

If that is done then this plugin cna be freed up to non-core experiments like this.

I should add that the AddAutoViewTemplatePlugintoFoswiki feature does not appear to have been completed, as there are no unit tests, and topics such as SkinTemplates don't mention the new functionality.

-- SvenDowideit - 08 Dec 2009

Michael, you are absolutely right about the %SET approach. On the flip side of that coin, the approach of keeping the rules in configure is likely to bite; I have several clients on hosted services who do not have access to configure. and would be unable to use this plugin. Interestingly enough I have a similar dilemma with my MirrorWebPlugin (not released yet); I ended up storing the rules for that plugin in a disk file, which is outside the Foswiki data tree but modifiable by someone with FTP access.

BTW I agree with Sven, with his comment above. I don't think this plugin should have been brought into the core. If it was outside the release process, you wouldn't be constrained this way. I accept my share of the blame for not realising this when it was first proposed as a core plugin. At the end of the day, I don't disagree with the concept of what you are trying to do - just the implementation.

-- CrawfordCurrie - 08 Dec 2009

So another outcome of this discussion could be that AutoViewTemplatePlugin is removed from the core again? I'd be fine with that!

-- MichaelDaum - 08 Dec 2009

I was going to vote neutral, as there are very good reasons for and against. But I think SvenDowideit's suggestion that code could be taken out of the plugin into the core, freeing up AutoViewTemplatePlugin for Michael's work where the idea can be safely developed and tested is a more attractive option.

-- PaulHarvey - 08 Dec 2009

I just realized that what this proposal is trying to aconplish is to to build an artificial category for a topic using patterns of web and topic names, and then use that category to choose the VIEW and EDIT templates. So far the only way to categorize a topic is using a form. If we want to keep thing simple, and use only forms to categorize, then what MichaelDaum want to acomplish with SolrPlugin and WebSearch can be achieved by adding the proper form to shipped WebSearch topics. But, for that to properly work, AutoViewTemplatePlugin must be modified to have a search path for the templates... which is something the core already does.

• That's what I observe regularly: users attach a bogus form just to get the automatic view template on these topics. In most cases the same effect can be accomplished differently, i.e. assigning a view template in a static way. -- MD
• A similar idea to this (for WebSearch) has been proposed in UpgradeSafeCustomisationsOfSystemTopics, but unfortunately it has stalled due to lack of agreement. -- AndrewJones
• Yep, that's where the idea for this proposal originates from. Means: if we AddAutoViewTemplatePlugintoFoswiki and decide to have RulebasedViewTemplates then we get UpgradeSafeCustomisationsOfSystemTopics. -- MD

So another alternative would be to update the AutoViewTemplatePlugin to let the core search for the proper template (the plugin will only "build" the proper name). This is way simpler from the user point of view.

(I'm not sure how AutoViewTemplatePlugin looks for the template, so perhaps no additional coding is required)

• It doesn't. Searching for the concrete view template was always a matter of the core. See Templates::readTemplate() . AutoViewTemplatePlugin only makes use of these standard mechanism by tweaking the search path dynamically for each topic a user visits. -- MD

-- RafaelAlvarez - 09 Dec 2009

Yes, I think many of us tend to think AddAutoViewTemplatePlugintoFoswiki hasn't been finished properly, and that its completion should be the implementation of the current set of functionnalities (choosing a ViewTemplate based on the form attached to a topic), should be integrated into core. Once this is done, AutoViewTemplatePlugin can be freed again as a non-core plugin and Michael can freely publish his changes.

-- OlivierRaginel - 09 Dec 2009

Adding the AddAutoViewTemplatePlugintoFoswiki to core or keeping in plugin. I would not care.

I would care if this EXCELLENT GEM of a feature was removed and put back in a plugin that I cannot rely on being always up to date each time a new foswiki release is done. With rely on I mean both that the spec/behaviour is under feature proposal control and that bugs become release blockers and will always be fixed before a foswiki release.

I am sure I am not the only one that has already implemented mission critical applications with this plugin based on the knowledge that I can count on it as a default stable feature from 1.1 and forward. The AddAutoViewTemplatePlugintoFoswiki was raised in June and after a good debate we reached clear consensus in October. We should be able to trust the decisions we make when we spend so much time on it. I love this new feature. It has really filled a big gap I had. I love this new feature.

So please do not remove the form based feature again. We are discussing if the additional configure based rule feature should be added as Michael has suggested it or not. If the answer is yes Michael checks it in. If No things are not over. Michael can fix his personal need with a new plugin and disable the default plugin. What Michael needs is a decision now so he can get on with a job. I think after more than 5 weeks of debate we should be able to make a decision on such a simple feature.

If the result of the vote is a NO we can also forward new proposals, with a spec that is more acceptable to a majority. From what I can read of the new reactions it is not the functionality that is the problem. It is how the spec is defined which created a number of issues (geeky syntax, only configure, interaction with already complicated template/skin system, etc etc).

If someone wants to merge the already existing plugin feature into core - go ahead. But please keep the spec intact. Checking how the plugin works this should not be a problem. Only the configure settings need to change name. Anything done in topics can work just the same if implemented in core. But again. This proposal is ONLY about Michaels proposal. Yes or No to Michaels proposal. If people start mixing in 10 other counter proposals we end up not knowing what we vote for. If you want a counter proposal raise it as new proposal and vote No for this one.

Someone misinterpreted my statements above claiming that I wanted to see a plugin code first to make a decision. I don't need that. Michael's spec is clear and easy to understand.

-- KennethLavrsen - 09 Dec 2009

Okay, voting period is over. The proposal has been rejected with 8 pros and 3 cons. That's very clear. The AutoViewTemplatePlugin will be forked into a non-core plugin called AutoTemplatePlugin containing the attached code.

-- MichaelDaum - 16 Dec 2009