EnableWikiEditingOfSystemTopics

Enable Wiki editing of System topics

We are "the wiki that can't be edited" because of our SVN based development of the System documentation topics. This acts as a barrier preventing contribution from web users. We also don't have a way to get feedback for System topics.

Improved editing has been previously proposed, but never fully designed and implemented. (or if it's been designed, some of this has been lost).
  • CrawfordCurrie wrote tools/mergeDoc.pl to handle merging of topics to/from SVN.
    • It assumes that the SVN tracked part is enclosed in =%STARTSECTION{"distributiondoc"}%
    • It allows discussion and other notes outside of the SVN tracked section.
  • SvenDowideit added DocumentationGroup to permit limited editing of System topics.
We have two different deployments.
  • trunk.foswiki.org is a run-from-SVN checkout
  • foswiki.org is a traditionally installed static installation.
Do we want to allow editing of Trunk, Release 1.1. or both?

We allow translations to be committed by WikiTranslationGnome. We probably want to add a WikiDocumentationGnome to commit web document changes back into SVN.

Is this a bi-directional integration with changes permitted in both SVN and Wiki?
  • How are conflicts handled?
Cron job to periodically merge? AfterSaveHandler to commit to SVN?

Do we allow edits of just Core System topics? Core Extensions? Core and non-core?

From IRC discussion

Assumptions

  • Most of the time, the Trunk and Release 1.1 versions of topics are the same, or have accidentally diverged.
  • Many would prefer wiki editing of topics, but svn edits will continue to occur.
  • Occasionally there are major restructuring of topics, but that is fairly infrequent.
  • SVN source of topics can be in multiple places - ie topics supplied by Extensions
  • Okay to sync one topic to multiple SVN locations. But not many to many!

Implementation

  • Create a single Documentation web. It has a copy of any topics intended to be web edited.
  • Each topic has form data:
    • Location: One or more SVN files that sync to this topic.
  • cron job
    • sync web and svn - how to deal with conflicts?

Implement a feedback mechanism

In addition to allowing topic edits, we need a way to gather comments and other feedback on System topics for users that don't have "commit authority" I'm proposing a ViewTemplate approach to gather comments on a separate topic for each System topic.

%TMPL:INCLUDE{"view"}%
%TMPL:DEF{"formtop"}%%TMPL:END%
%TMPL:DEF{"formbottom"}%%TMPL:END%
%TMPL:DEF{"attachmentsattop"}%%TMPL:END%
%TMPL:DEF{"attachmentsatbottom"}%%TMPL:END%
%TMPL:DEF{"simpleheader"}%\
<div style="margin-bottom:1em; padding:1em; background:#99ddaa;" >
<p>%ICON{warning}% This is an official distribution version of *<nop>%TOPIC%*. and is managed by Subversion revision controls.</p>
<p>%ICON{support}% Members of the Main.DocumentationGroup can edit this document.  Anyone is welcome to submit feedback, suggestions or other comments under the Discussion tab.</p>
</div>
%TMPL:END%
%TMPL:DEF{"bodyclassname"}%patternViewPage%IF{"'%TOPIC%' allows 'CHANGE'" then=" foswikiHasChangePermission" else=" foswikiHasNoChangePermission"}%%IF{"context authenticated" then=" foswikiLoggedIn"}%%IF{"defined raw" then=" patternRawViewPage"}%%TMPL:END%
%TMPL:DEF{"content"}%
%TMPL:P{"simpleheader"}%
%TABPANE{select="%URLPARAM{"tab"}%"}%

%TAB{"Documentation"}%
%TEXT%
%TMPL:P{"form"}%
%ENDTAB%

%TAB{"Discussion" id="discuss"}%
#FeedBack
---++ Feedback and suggested changes
%IF{"istopic 'Sandbox.%BASETOPIC%'" then="$percntINCLUDE{\"Sandbox.%BASETOPIC%\"}$percnt" }%
%COMMENT{target="Sandbox.%BASETOPIC%" type="return"}%
%ENDTAB% 

%IF{"%QUERY{"length(attachments[NOT lc(attr)=~'h'])"}% > 0" then="
$percntTAB{$quotDownload$quot}$percnt
$percntTMPL:P{$quotattachments$quot}$percnt
$percntENDTAB$percnt
"}%

%ENDTABPANE%
%TMPL:END%

The template works with CSS to disable the Edit button for users without edit authority. (CSS had significant help from ArthurClemens) (Add this to base_css)
.foswikiHasNoChangePermission.foswikiLoggedIn .foswikiRequiresChangePermission { display: none; }

We need an alternate version of the "return" comment template that includes ?tab=discuss in the return url. The above template collects comments into the Sandbox web.
  • Collect comments into System/Feedback subweb?
  • Same topic name. or %BASETOPIC%Feedback
  • How to control which topics get the SystemViewTemplate (ie Not WebPreferences)
-- GeorgeClark - 13 Jan 2012

How about using MetaCommentPlugin and strip off any %META:COMMENT meta data on svn checkin? Having comment shadow topics is always a bit tricky when the original topic is renamed or moved. Having all in one place might be more convenient.

-- MichaelDaum - 13 Jan 2012

This business of creating "documentation topics" is what MediaWiki is strong at (Wikipedophilia), so let's take a lesson from them. The idea of having a "documentation topic" and a "discussion page" is something that should be fairly easy to implement in Foswiki using a WikiApp alone (i.e no new plugins on f.o). I think this is the direction George is driving in above.

-- CrawfordCurrie - 13 Jan 2012

Yes. I thought some about MetaCommentPlugin, but I wanted the additional flexibility that the discussion page could be more fully edited by anyone. The comment box was there to accomplish a couple of things - make it as easy as possible for a new user to provide feedback, and to seed the first comment without needing to create all the feedback topics in advance. The template still needs some work to add a "Edit the Feedback" button.

Also, the fewer changes made to the core system topic, the merging to/from SVN will be simpler and less frequent.

-- GeorgeClark - 13 Jan 2012

Discussion from IRC:

gac410
a SvnGnome?
ugh - and CDot's merge. I wonder, for 1.1.4, can we be uni-directional. Updates done on System web feed into svn, but not the other direction?
Babar
like a WikiTranslationGnome smile
gac410
yeah.
Babar
not the other way around will be hard...
that implies nobody will ever commit anything in the core/data/System directory unless it's the gnome
can be done, but seems cumbersome to say the least
gac410
yeah. The challenge is to allow wiki updates to System docs, so there is no SVN hurdle for the DocumentationGroup. Maybe this is why Sven stopped when he did.

gac410
CDot - Regarding mergeDoc.pl - how did you plan to handle collisions - edits in both SVN and Wiki?
-
gac410 not all that good at reading / understanding code sometimes.
CDot
gac410: I didn't. I was assuming that the wiki would become the master.
c.f. pootle
gac410
ugh. yeah, I suspect that will get some pushback. I sort of started down that path as well. Can't recall Babar's comment now.
Babar
can't recall my own comment either smile
oh right, I wanted auto-commit from the web
-
CDot checked the code and found # update twiki from subversion
Babar
or impossible to commit to subversion
gac410
http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2012-01-13,Fri&sel=6#l2
CDot
so it updates the merged version to the web as well
gac410
Right - it can be run by commandline either one direction or the other.
CDot
right
gac410
so which one changes first could trigger a merge. But if both change big grin
CDot
gac410: more tricky is when the same line is changed in both; conflict marks could be a PITA to eradicate
CDot: I HOPE by restricting the editing of System topics to a smaller group of Documentation editors that prefer to not use SVN, the likelyhood of collisions is reduced.
CDot
gac410: agreed. I would most certainly prefer to use the wiki.... though, what happens when a doc change is made that should really apply to ReleaseNNxMM and trunk?
do I have to do the wiki edit twice, in two different webs?
CDot
how does a "normal" doc author know to do that?
gac410
CDot: yeah - this is a rather challenging issue - I suspect that's why these efforts keep stalling.
And is also why I went from "charging ahead" to "need a brainstorm topic"
CDot
well, let's see. Most doc topics are identical in trunk and release. Most changes will apply to both
gac410
CDot: yes. I'd say we are in sync more than we diverge. Except for major refactoring like new QUERY operators, etc.
CDot
so, let's say we compared the trunk rev with the release rev before merging the changes
if they are identical => trivial merge
if they differ, then simply add a note to the trunk topic to say it needs TLC
or even (gasp) auto-create a task
gac410
Hm. Auto-create tasks - that could get busy. ... I wonder how many topics are different today. probably not a simple question due to TOPICINFO timestamp.
hm... I have a find / git diff master: release11: script somewhere.
find core/data/System/ -type f -iname "*.txt" -exec git diff master:{} release11:{} ; Mostly it's TOPICINFO differences
-
CDot does a diff and finds quite a few betternesses in trunk
gac410
QuerySearch Sandbox and SEARCH macro restructuring.
SystemRequirements
CDot
y
gac410
TBH I think you are on to something. Assume that all changes apply to both, Maybe add a "* Set DONOTMEREGE =1" for topics where we want to have different paths - like QUERYSEARCH
CDot
anyway, after an initial bit of pain, I suspect they'll settle down to be pretty near identical
short of a major doc restructuring, that is
would be nice to feed back into the doc topic on f.o if they differ
gac410
Major doc restructuring maybe could be done in a separate web, and then merged as a bulk change once the restructuring is completed.
CDot
WARNING! This topic is not copacetic!
gac410
Plugin for F.O - compare MD5 from trunk and release?
or just simple diff
CDot
simple diff works for me
kiss
gac410
add it to the view template for System that I've started on.
Actually could be a %TAB that presents the diff from trunk so easy to see the differences. Too bad tabs couldn't be conditional
%IF (difference found with trunk) %TAB <show the diff>
CDot
that would be most excellent
gac410
As is, could easily have %TAB %SVNDIFF% or %TRUNKDIFF% or something like that.
I'd prefer to have the diff rendered in a raw format - makes it much easier to see macro differences.
(Actually i'd *really* prefer if all topic diffs could be raw format.)
Actually, present the diff in a format with a save-as-file button so I could just apply the diff to a checkout.
-
CDot worries that gac410 is at risk of over-engineering
gac410
I suppose. Though if taking a diff to return true/false not much of a leap to return the diff.
save/download is probably not needed.
but that's the point of brainstorming.
Now for a simple plugin. Paths stored in configure params to prevent any fishing outside of expected directories. two macros .. 1) return true/false, 2) Return actual diff inside a <verbatim> block. Exclude TOPICINFO from comparison.
CDot
why do you need a plugin?
gac410
How else to compare trunk/System/blah with release11/System/blah ??
and show the difference or even a "This topic is dfferent" warning
-
gac410 is missing something I suspect? He is ignoring the background merge challenge, instead working on a view template that shows the differences.
CDot: Did you understand - am I wandering off in a different path?
CDot
gac410: hmmmm?
erm, well. There are two checkouts on the server. Just diff the files.
gac410
right exactly. But how do I show those diff's in a tab of the view template without a macro.
%IF (difference found with trunk) %TAB <show the diff>
(11:12:13 AM) CDot: that would be most excellent
CDot
I had in mind that your offline script would poke that message into it somehow
perhaps in the discussion topic?
in a %SECTION then %INCLUDEd into the template
sort of thing
gac410
that seems harder somehow.
CDot
harder? you think? ok.
gac410
The less updating offline the better
-
CDot disagrees; update offline => 1 diff. In a plugin => a diff per view.
CDot
verrrrry expensive
and if you cache the diff..... well, why not just do it offline?
gac410
servers get faster, brain gets slower
CDot
it's the temperature. Brains work better when they are hot.
gac410
I figured the cron script would be resolving diff's and committing. ... no, that's between Web and SVN. This is between Web and Web. need a 4-way diff. gac410 melts down.
So within a branch, script syncs between SVN and Web. Trunk has challenge that it already runs from SVN, I think that makes it harder
between branches - I figured that would be a macro. but yes that gets expensive. so need a 2nd script that does a cross-branch diff and stashes the results somewhere.
Probably just replace a differences topic when diff found as you say it can be included into a tab if it exists.
<verbatim> `diff -Naur release trunk` </verbatim> but a bit more complex to exclude the TOPICINFO
CDot
why do we need to diff Web/Web?
seems to me safer to always refer back to svn
a general-purpose Topic-Topic diff would take care of the in-wiki requirements - and that would be a plugin
gac410
So the diff we show is the is the branch to trunk diff, which won't necessarily correspond to the web view of the topic ... assuming topic has been edited and sync has not yet run?
I guess it depends upon the flow. Do we expect that most devs would just use the wiki and let it auto-commit back?
We have 4 copies. trunk, trunk svn, release, release svn. the wiki<->svn diff will be transitive. trunk<-> release either accidental or intentional.
(right now 3 copies as trunk gets overwritten by svn.
Can trunk<->release be automatic, or do we need a "Sync to trunk" button so that developer can make intentional decision to sync the two versions.
not trying to make this hard, but it's complex enough that I suspect it could easily fall apart.
CDot
why do we need 2 webs?
Babar
hum... trunk<-> release cannot be automatic. Sometimes they can't be
like.. new features in trunk, new behaviors, new config...
CDot
the way I see it, only 1 web is needed, showing the "latest docs".
gac410
right. I figured we'd need a "DoNotSync" setting.
CDot
i.e. the current release docs
anything more "exciting" will be done in svn
cos more exciting => someone is hacking code too
(of course this theory collapses if people start creating new doc topics, but....)
gac410
So today we have trunk.f.o and f.o ... each have System web. Trunk is SVN mirror. f.o is a static copy. Does one of them become the doc web that is updatable?
-
CDot reckons that 99.99% of doc edits are tweaks to existing docs
gac410
agreed
CDot
the doc web on trunk - System - is already a checkout
so will reflect any changes made to it via a "release" web doc edit
gac410
yeah - that's what I meant by SVN mirror - poor terminology choice
CDot
or in fact, we don;t want this editing to be done in the System web, do we?
cos that's a checkout area
so, lets say we had a "Documentation" web which is the web we've been discussing
that gets synched back to the "System" web in trubnk and the release branch
gac410
Yeah, that makes it difficult. I ws envisioning edits on f.o System web with script to sync with a shadow svn checkout of just data/System
CDot
job done smile
difficult compared to editing topics in the System web of a pseudo-install?
cos f.o and t.fo. are just conventional pseudo-installs
gac410
No. f.o is a install from tarball. t.f.o is the pseudo install
CDot
also, if you allow edits of System web you allow editing of Plugin topics too
gac410
yeah. I was carefully ignoring that issue. Thanks big grin
CDot
my pleasure smile
of course it would be neat for a plugin author to say "allow my plugin topic to be edited online"
gac410
So... let's ignore System webs for now. Create a "Documentation" web which is the master copy to be synced with SVN.
CDot
esp. since most Support q's are related to plugins
y, and bear in mind it might synch with a topic in different webs
i.e. Documentation might have a topic that links to - e.g. - CommentPlugin.txt in CommentPlugin/data/CommentPlugin.txt
maybe that's a bridge too far.
gac410
no I don't think so - it addresses the plugin issue
Do docs in "Documentation" web need meta that says where they sync from/to
If we add a Form to system docs, it makes the view template easier to apply.
Or at least when Documentation is created, each topic gets a form with SVN location, Flag to control sync trunk / release branch.
Hm. Maybe creating a topic in Documentation is the controlling event. If you want online edit of your Plugin topic, you copy it into Documentation with form data saying where it lives
So "System Web" gets flagged with "This is a static copy of Release Blah ... for latest documentation see Documentation/EnableWikiEditingOfSystemTopics"
CDot
I guess so, yes. What we are creating is a "bucket" on f.o where we can throw topics from anywhere on svn to have them synched
not just that, but the bucket can synch to more than one place (N places, in fact)
gac410
Can we go as far as to say "If you put things in this bucket, strongly suggest not updating in svn"
CDot
that all requires meta-data to explain to the tools what to dio with the topic
well, you cabn strongly suggest, but you won't stop me I'm afraid
as updating in SVN is much, much faster for the likes of me
gac410
right. Maybe this could finally solve the old T* issues with web per release.
CDot
hmmm. Not sure about that. Of course if there is a major doc shift between releases, we can always create a second Documentation web and point the meta-data differently
gac410
That's why I worded it as a suggestion smile
CDot
smile
gac410
Does it have to sync with identical topic name. Or can meta contain target by name. Documentation/QuerySearchNew syncs to trunk/core/System/QuerySearch
Rather than requiring a whole new web to handle major changes.
And what to do with rcs ,v file ... When sync to SVN completed, discard the ,v?
CDot
you mean when the topic is updated from svn? you shouldn't have to do anything
the ,v will be auto-updated next time the topic is edited (as per our more recent fixes for that)
gac410
okay.
CDot
golly; wouldn't targeting multi-topics to a single svn entry get a bit.... fraught?
one topic -> many svn is ok, but the reverse? mummy! my brain hurts!
gac410
Wasnt suggesting that. More that if you want to diverge topic1 <-> trunk/Topic1 and oldTopic1 <-> branch/Topic1
vs. creating a separate documentation web
For example, QuerySearch is major changes trunk to release. That's one we don't want to sync between branches, but still might want the wiki editing.
ie does a single major doc change require your comment: if there is a major doc shift between releases, we can always create a second Documentation web and point the meta-data differently
I'll add this discussion to the topic, and then try to capture / summarize it in the document. one more question. Do we set it up to redirect System -> Documentation
If Documentation web is the master copy feeding both trunk and release11, for most topics anyway, is the idea of a diff still useful, and what is being compared.
CDot
oic; well, if it's constrained to that simle case (no many-one topics-svn) then no probs
the idea of a diff is useful, yes; but a generic diff between topics
-
CDot smells fresh blood, so has gone to feed
gac410
yes. no sense creating versioning hell. Right. Once Documentation web exists, maybe the diff is to the original static (System web) copy?
CDot
good ploy

Topic revision: r8 - 09 Jan 2015, GeorgeClark - This page was cached on 03 Mar 2021 - 06:45.

The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy