Release Meeting: 21 Sep 2015

(09:00:01 AM) jomo  entered the room.
(09:04:35 AM) gac410: hi everyone.   Sorry I'm late.  Lost track of time
(09:05:32 AM) gac410: Agenda for meeting:  http://foswiki.org/Development/ReleaseMeeting02x01_20150921
(09:06:54 AM) CDot  entered the room.
(09:07:03 AM) CDot: hadn't realised the time
(09:07:19 AM) gac410: me neither  :)
(09:09:10 AM) gac410: Hi everyone.  ...   lets get started.   Everyone awake?
(09:10:18 AM) jomo: d:) 
(09:10:18 AM) gac410: On today's agenda.   Blockers for 2.0.2,   and hopefully we'll get to some feature reviews for 2.1
(09:10:21 AM) MichaelDaum: hey guys
(09:12:13 AM) gac410: Okay Blockers ...  Kenneth went through the full set of EditTable and Table tests from the Testcases web,  and opened up 10 issues, 8 of which he classified as a blocker.  
(09:12:31 AM) MichaelDaum: http://foswiki.org/Tasks/PatchReleaseBlocker
(09:12:51 AM) gac410: I also listed them in   http://foswiki.org/Development/ReleaseMeeting02x01_20150921
(09:13:50 AM) gac410: There is one not related to EditRowPlugin ... let's deal with that first.    http://foswiki.org/Tasks/Item13696
(09:14:54 AM) gac410: turns out Foswiki::Attach always inserts pub URLs into topic as utf-8 and entity encoded.    So non-ascii attachments are not reachable even on a clean 2.0 install
(09:15:56 AM) MichaelDaum: I will remove the local NameFilter impls in ImagePlugin and TopicInteractionPlugin as soon as possible 
(09:16:17 AM) MichaelDaum: so any fix in core also makes it into these :(
(09:16:18 AM) CDot: gac410: don't understand why your plugin can't deal with encoded uris
(09:16:24 AM) gac410: So the url issue is not just a migration issue,  assuming anyone would actually run a non-utf8 store if not migrating.
(09:16:55 AM) gac410: CDot ... it can.   Just wanted to discuss and agree that's how we fix.
(09:17:15 AM) CDot: absolutely it's the right way to fix it.
(09:17:50 AM) gac410: So proposal.   Keep Foswiki running as utf-8 including in inserted topic links,  and let the PubLinkFixup plugin handle bad links in a complete page handler.
(09:18:14 AM) CDot: assuming that there are people out there brave enough to run with {Store}{Encoding}. Personally I think it's a terrible idea.
(09:18:44 AM) favioflamingo entered the room.
(09:18:55 AM) gac410: Lavr is now converting, but it seems andreli isn't planning on converting - that's IBM
(09:19:46 AM) jast entered the room.
(09:19:47 AM) Lavr: Kenneth arrived at meeting Hi guys
(09:19:54 AM) gac410: Hi Kenneth
(09:19:57 AM) CDot: I see that as a huge legacy burden, that has potential to cripple the future.
(09:19:57 AM) MichaelDaum: not IBM. he is at http://www.psi.ch/
(09:20:04 AM) gac410: Oh... okay
(09:20:14 AM) MichaelDaum: lots of webs - lots of people. heavy lifting.
(09:20:28 AM) gac410: oops.   I thought he said IBM,  must be confusing someone.
(09:20:45 AM) CDot: Colas is IBM
(09:21:05 AM) MichaelDaum: as is Carlo Schulz
(09:22:29 AM) gac410: okay.  So to summarize.   utf-8 internals (but NOT TinyMCE) always generates entity encoded utf-8 pub URLs.    So even if store is iso-88,  we continue to use utf-8 wherever possible and use the plugin to fix up links we detect as pub links.
(09:23:25 AM) gac410: In that case I think http://foswiki.org/Tasks/Item13696   is ready to go WaitingForRelease.
(09:23:48 AM) CDot: :-)
(09:24:09 AM) gac410: So now to our loverly can of wurms.     ... EditRowPlugin
(09:24:28 AM) CDot: gac410: when you say "not TinMCE" do you *mean* TMCE or do you mean "The TinyMCEPlugin"?
(09:24:42 AM) gac410: TinyMCEPlugin doesn't encode inserted links.
(09:24:56 AM) CDot: so you mean TinyMCEPlugin?
(09:25:31 AM) gac410: So you end up with raw utf-8 urls.      I think so, yes.  TinyMCEPlugin.   I think it's a foswiki tmce plugin which inserts attachment links
(09:25:41 AM) gac410: in the javascript code.
(09:26:08 AM) CDot: ok. Because *that* can be fixed.
(09:26:30 AM) CDot: if it was in the TMCE code, then I wouldn't touch it.
(09:26:59 AM) ***gac410 has learned not to fiddle too much with TinyMCEPlugin or WysiwygPlugin.   Always regrets it :(
(09:28:13 AM) gac410: Okay.   So,  EditRowPlugin.  As an alternative solution to get 2.0.2 out with all the ERP blockers,  I added a task to put EditTablePlugin back in the MANIFEST.  But not enabled by default.
(09:29:08 AM) Lavr: I cannot understand - given all the very severe bugs I have reported - that you want it enabled by default.
(09:29:36 AM) gac410: The failures are all in existing test cases.  So if I had done due diligence and run the test cases web completely,  we never would have released 2.0
(09:30:21 AM) CDot: TinyMCEPlugin/pub/System/TinyMCEPlugin/plugins/foswikibuttons/jscripts/attach.uncompressed.js:42 is what you want. Just URL-encode.
(09:30:50 AM) gac410: What's others opinions. ...  Lavr says flip back to ETP as default.   Any other opinions? 
(09:31:11 AM) ***CDot refuses to have an opinion. Will accept whatever is decided.
(09:31:35 AM) ***gac410 tends to try to move forward with the new extensions,  and not fall back too far Lavr ..   
(09:31:53 AM) Lavr: CDot you observed a ETP crash that I did not see. My guess is it is related to a bug that George fixed this weekend (new Perl version)
(09:32:22 AM) CDot: Possibly. it was a multi_param problem, IIRC.
(09:32:50 AM) gac410: There were two bugs.    ETP used unescaped braces  (perl 5.22)   and called param in a list context.   My fix to that one assumed that it really did want a list.
(09:33:13 AM) gac410: Both fixed and unit tests pass on a 5.22 perl system.
(09:34:09 AM) Lavr: ERP as it is now destroys people's content. Edit a page a little and you have a major clean up to do. I mean. I had a test topic with more than 1000 lines of garbage after few minutes of normal editing of the tables. Shipping with this enabled will give Foswiki a bad reputation.
(09:34:23 AM) gac410: CDot  have you had a chance to review the bad news ... any hope for fixes?
(09:34:40 AM) Lavr: ETP may be old but it works. And ERP can be reenabled after a simple plugin update when we have the fix available
(09:34:42 AM) CDot: Just a reminder. The original rationale for the move to ERP was that ETP was unmaintained, and no-one was interested in maintaining it. ERP was under development, and looked like it could fill the gap.
(09:34:43 AM) gac410: Lavr,  is that because it expands macros?
(09:35:15 AM) CDot: Now, it ETP is considered to now be maintained, and is more functional than ERP, I am quite happy if you want to drop ERP from the roster.
(09:35:22 AM) Lavr: I don't know what goes wrong. It makes 1000s of copies of the TABLE tag line in some of the trials I did.
(09:35:57 AM) Lavr: Even though I opened many bug items that seem related and probably have a common root cause
(09:36:16 AM) gac410: CDot:   we really need it to be reasonably compatible for the install base.       I would NOT call ETP maintained.   I fixed some really basic perl compatibility bugs
(09:36:37 AM) Lavr: I just opened many cases because the exact symptom was not the same. And I wanted to make sure ALL the test cases were tested and tried before the bug is declared fixed
(09:36:46 AM) CDot: I can't promise fixes just like that. It can take hours to analyse those testcases. Just establishing what the tasks are reporting is proving difficult, as several just say "the testcase fails" without saying in what way.
(09:36:47 AM) gac410: The few times I've looked at ETP (or ERP for that matter),  I've become totally lostl
(09:38:38 AM) Lavr: I think shipping ETP enabled is the right action. And keep ERP in the release disabled so it is easy to enable when we have had good long time to make a fix and test it 100%
(09:39:26 AM) gac410: Any other opinions?   
(09:40:24 AM) gac410: Lavr, (or anyone...)   have you tested ETP with utf-8 data?  
(09:40:33 AM) jast: I can't comment, I've hardly ever used either of the two enough to trip over _anything_
(09:40:44 AM) Lavr: yes. I tested AFTER I did the conversion and at home I am running UTF-8 only
(09:43:25 AM) gac410: There is only one ETP bug in the backlog I'd consider at all serious.  ETP fails on fastcgi sites when used in an included topic.   Most of the rest are interoperation with non-default extensions. or enhancement requests.
(09:43:36 AM) Lavr: Just tried now to put some non ASCII letters in an EDITTABLE table and they are stored correctly in UTF-8
(09:45:09 AM) jast: small reminder: all non-ASCII characters are not created equal. those with a code point below U+0100 can often work even if full Unicode support is not implemented correctly
(09:45:57 AM) gac410: Okay.   No other opinions...   I'll make ETP the default, but leave ERP in the release.   It still has big benefits  client side sorting, etc.  Will have to juggle some of the checker warnings.
(09:46:05 AM) jast: I've taken to testing with Greek characters, uppercase starting at U+0391 and lowercase starting at U+03B1
(09:46:21 AM) gac410: That will let us get 2.0.2 out hopefully reasonably soon.
(09:46:54 AM) gac410: The ERP issues will become blockers for 2.0.3 and 2.1 
(09:47:43 AM) gac410: Okay    any other blocker discussion?    Can we spend  15-20 minutes on 2.1 feature requests?
(09:48:09 AM) MichaelDaum: yes please
(09:48:10 AM) Lavr: I have no more release blockers
(09:48:54 AM) gac410: Okay.   I copied the list of all features "tagged" at planned for 2.1,  into http://foswiki.org/Development/ReleaseMeeting02x01_20150921
(09:50:27 AM) JulianLevens entered the room.
(09:50:36 AM) JulianLevens: Hi
(09:50:39 AM) gac410: Hi JulianLevens  Thanks.  
(09:50:42 AM) MichaelDaum: most probably we will see more changes, i.e. on JQueryPlugin
(09:51:20 AM) MichaelDaum: I'd like to remove some cruft in there as well as update 3rd party libs from upstream
(09:51:38 AM) gac410: We can add more,   They just go on agenda for another meeting.   For the ones we have,  need to decide what will not be implemented.
(09:52:26 AM) gac410: First up ... http://fhttp://foswiki.org/Development/AddConcatOptionToAttrs ... JulianLevens ... are you okay to get that into 2.1 ?
(09:52:36 AM) JulianLevens: y
(09:53:08 AM) gac410: Ah.. .maybe we should pick a target.    Where are we targeting 2.1?    December too aggressive?   
(09:53:34 AM) gac410: Jan 4th would be 6 months from 2.0 release.
(09:53:36 AM) MichaelDaum: probably yes. but hey. better keep pushing.
(09:54:06 AM) gac410: If we follow branch/merge, and don't merge until reasonably tested,  we can easily defer features that don't make the target
(09:54:14 AM) MichaelDaum: gac410, maybe we can implement the notion of a merge window where new features must have been started ... or are deferred.
(09:55:03 AM) MichaelDaum: feature freeze we called it
(09:55:26 AM) Lavr: Always a good strategy
(09:55:27 AM) gac410: hm   That,  or maybe a little stronger.  Not just "started"  but feature "ready to merge" says that it's reasonably ready to release while sitting in a branch.
(09:55:50 AM) MichaelDaum: okay
(09:56:15 AM) gac410: Historically we've had too many started features that don't really hit feature complete.   Much easier on the RM anyway to merge features once reasonably complete.
(09:56:21 AM) gac410: ;)
(09:56:46 AM) MichaelDaum: as long as new features go into a feature branch we will be able to implement such a strategy
(09:56:47 AM) gac410: Okay,   Next up http://foswiki.org/Development/ConfigurableURLIncludes ... MichaelDaum    Plan for 2.1?
(09:57:01 AM) MichaelDaum: yes. easy. nothign ground shaking.
(09:57:11 AM) andreli entered the room.
(09:57:18 AM) MichaelDaum: was too lazy to implement it since 1&1 asked for it
(09:57:20 AM) gac410: Yes please.  I'd like to keep the master branch 1) reasonably ready to release, and 2)  reasonably stable for anyone who downloads from github.
(09:57:51 AM) gac410: Next one I decided to own from pharvey.  http://foswiki.org/Development/DeprecateHTTPandHTTPS     I'll plan for 2.1
(09:58:25 AM) MichaelDaum: gac410, maybe it makes sense to move these two into a plugin of their own to ease migration
(09:59:00 AM) gac410: Proposal suggested just a config switch to disable them.  
(09:59:21 AM) MichaelDaum: I'd rather see the code being removed from the core and parked into a plugin
(09:59:21 AM) Lavr: I never used them. what are their typical purpose?
(09:59:31 AM) gac410: Then once deprecated, we can remove them in a subsequent release following deprecation process.
(10:00:29 AM) gac410: If we follow deprecation, they just go away,  assuming we really follow the process.   Then we don't really need a plugin hanging around.
(10:00:44 AM) gac410: Lavr,  I really don't know why. 
(10:01:42 AM) Lavr: They seem to be more for debug than for development of wiki apps.
(10:02:21 AM) Lavr: And for debug I can just look at the source. But whoever added them must have had an idea. Disable via configure is a good solution
(10:02:22 AM) gac410: pharvey's proposal was to whitelist arguments.   Maybe I should re-open this for discussion.  I lean toward simple deprectaion followed by removal  
(10:03:00 AM) gac410: If someone really wants them, they can add them in to a plugin when we complete deprecation with remove.
(10:03:22 AM) gac410: I'll re-open the timer on that one so we can discuss.   Let's move on.
(10:03:50 AM) Lavr: I can imagine controlling content based on browser as a feature for an end user.
(10:03:57 AM) Lavr: User-Agent
(10:04:39 AM) gac410: http://foswiki.org/Development/MakeZonesLessIntrusive   I don't fully understand one of the comments in the proposal so I can't just take ownership.   I'll defer unless someone has suggestions on how to implement
(10:06:02 AM) MichaelDaum: this proposal is about removing the implicit expansion of the head and script zone
(10:06:05 AM) gac410: MichaelDaum:   Your comment about adding {AutoRenderStandardZones} ... I didn't understand how that would fit with the patch.
(10:06:34 AM) MichaelDaum: these are injected into the html page by magic at the end of the rendering process
(10:06:55 AM) MichaelDaum: when they haven't been expanded previously using %RENDERZONE
(10:07:12 AM) MichaelDaum: Paul proposed to remove this magic and require skins to always contain appropriate %RENDERZONEs
(10:07:28 AM) MichaelDaum: ... which currently maintained skins already do
(10:08:01 AM) gac410: So this would break compat with unmatinained skins  .. WidgetSkin, NuSkin, etc.
(10:08:25 AM) MichaelDaum: probably yes. they would need to switch on AutoRenderStandardZones
(10:08:28 AM) jast: fairly straightforward to fix, though
(10:08:45 AM) gac410: Okay.   I'll keep my name on that one.    
(10:09:07 AM) MichaelDaum: not a brainer imho
(10:09:55 AM) gac410: Next up:  http://foswiki.org/Development/ReduceImpactOfCGIDotPMinFoswiki    This is a rather major one ... changing all of our generated CGI:: rendering into Templates.  and moving CSS classnames from hardcoded to the templates as well.
(10:10:20 AM) MichaelDaum: there is some html5-ish requirement in here as well
(10:10:23 AM) gac410: I had started an implementation, but  it really needed to be redesigned.  Had some serious flaws.
(10:10:41 AM) jast: aren't Templates rather overkill for that?
(10:11:30 AM) Lavr: It will be a tough job making custom templates and not run into trouble when people rename classes all the time
(10:12:04 AM) MichaelDaum: I'll take actions of macros defined in Foswiki::Plugins ... and make PLUGINDESCRIPTION and family a bit more useful addign format/header/footer/separator parameters
(10:12:15 AM) gac410: Lavr,  The cases we are talking about are the CSS class names that are hardcoded in perl,  like in tables.  
(10:12:16 AM) jast: I kind of see the value of simplicity in CDot's html() helper
(10:13:10 AM) MichaelDaum: it really depends on the amount of HTML hard-coded into perl
(10:13:39 AM) gac410: MichaelDaum:    I'm lost on your comment  I'll take actions of macros defined in Foswiki::Plugins...
(10:14:44 AM) MichaelDaum: gac410, Foswiki::Plugins has got lots of CGI::html rendering ...and macros defined in there are rather useless as they are right now for anything more intelligent
(10:15:14 AM) gac410: Okay,  so for ReduceImpactOfCGIDotPMinFoswiki ...  Lets re-open discussion ...   It's an accepted proposal but seems like there is some disagreement on implementation. 
(10:15:51 AM) MichaelDaum: gac410, in how far?
(10:16:32 AM) gac410: jast thinks templates are too heavy,  CDot has his html helper ?
(10:16:33 AM) MichaelDaum: goal is to remove any use of CGI.pm for rendering html
(10:16:34 AM) CDot: Unless you are going to move all the hard-coded html into templates (BIG job, IMHO) I think some sort of xml() (or html()) helper is the best bet.
(10:16:58 AM) MichaelDaum: both makes sense
(10:17:04 AM) CDot: I did it on a branch some time ago, wasn't hard.
(10:17:17 AM) gac410: I have a branch that started that process CDot.    (moving CGI::...   to templates.
(10:17:25 AM) MichaelDaum: Foswiki::html() for the little markup here and there - templates for more thorough blobs of html.
(10:17:53 AM) gac410: Implemented a HTML::   class which was a replacment for the CGI::  methods but delegated the layout to a template.
(10:17:58 AM) CDot: note that some of the code is written the way it is because CGI caused generation problems, or had undesireable knock-on effects on tests
(10:17:59 AM) jast: I've written my own html()-like helper, sure wouldn't mind using something built-in
(10:18:14 AM) CDot: MichaelDaum: right.
(10:18:23 AM) jast: there's just one subtletly that seems to not be covered by this: some tags must have close tags, others must not
(10:18:28 AM) MichaelDaum: in addition
(10:18:40 AM) MichaelDaum: whenever a macro genertes html, offer use control
(10:18:42 AM) CDot: jast: not so subtle, that. And fairly easy to handle :-)
(10:18:49 AM) gac410: Okay.   So  I really think it's worth re-opening the timer on that proposal  and discussing it in-topic.      
(10:18:51 AM) jast: right. just saying.
(10:22:50 AM) gac410: Next:  http://foswiki.org/Development/SplitTopicAttachmentNameFilters     That one is fairly simple.   CDot,  I was more concerned that you don't like filter-out   but didn't want to tackle major changes to separate topic vs. attachment name restrictions.
(10:23:27 AM) gac410: Ie  leave the current filter-out strategy, just use a different filter,  rather than tackle the "reject don't rename" proposal.
(10:24:33 AM) gac410: Next is Michael's http://foswiki.org/Development/TopicDisplayName   ...  Are you in good shape on that one for 2.1?
(10:25:59 AM) MichaelDaum: yea
(10:26:38 AM) MichaelDaum: there has been very positive feedback by users of this feature already available via plugins
(10:26:58 AM) gac410: The last 3 features are "under investigation"   JulianLevens    http://foswiki.org/Development/AddBacklinksToQuery      That one addresses a huge utf-8 issue.
(10:27:03 AM) MichaelDaum: moving it into the core also needs a couple of plugins to change to use official TopicTitle apis 
(10:27:30 AM) MichaelDaum: http://foswiki.org/Development/PreventRedundantFileReadsInStore depends on nytprofile results and may result in other performance fixes
(10:28:33 AM) Lavr: redundant reads does not sound like a feature request
(10:28:48 AM) MichaelDaum: the title of this proposal might therefor be a little misleading ... as the required actions are already anticipated 
(10:28:49 AM) gac410: We have to be a bit careful of caching too much I think.    We have the MetaCache that search uses.   If we end up with a store cache, could we end up with multiple cached copies of topic data,   
(10:29:08 AM) MichaelDaum: gac410, xactly
(10:29:19 AM) gac410: Need to worry about memory growth in big search tasks,   and possibilities of stale cache.
(10:29:48 AM) JulianLevens: I should be able to manage AddBackLinksToQuery by December
(10:29:56 AM) JulianLevens: Is that Dec 1st?
(10:30:23 AM) gac410: That would be great JulianLevens.   Let's say merge ready by Dec 1st,  so we can work towards a Jan 4th final release?
(10:30:27 AM) jast: there are only two hard things in computer science: cache invalidation, naming things, and off-by-one errors.
(10:30:57 AM) gac410: very true jast
(10:31:04 AM) MichaelDaum: could be PreventRedundantFileReadsInStore renders an empty vessel for performace review/fixes
(10:31:51 AM) gac410: So everyone ... On the release plan,    I'll make the "ready for merge"   date 12/1 for 2.1,   and targets leading toward a 1/4/16 release.    
(10:32:24 AM) gac410: MichaelDaum:   Last one is also yours:   http://foswiki.org/Development/AddRegistrationTokenToUserinfo
(10:33:00 AM) MichaelDaum: this is valuable info stored in WikiUsers ... but not available to wiki apps
(10:33:05 AM) Lavr: For the none-yanks 12/1 means December first ;-)
(10:33:44 AM) CDot: 12th January, surely?
(10:33:44 AM) jast: ISO dates please ;)
(10:34:06 AM) CDot: 1st April is a great day for a release
(10:34:17 AM) gac410: And CDot ....   any thoughs on Normalization?    I have a patch that uses normalize for sorting ... isolated to the (a cmp b) code that seems to work     But we really have big normalization issues on osx
(10:34:38 AM) MichaelDaum: okay are we winds-down? gotta run and get some food before the next call.
(10:34:43 AM) gac410: I'd like to see a 6-month minor/major release cycle.    Keep project moving even if smaller releases.
(10:35:02 AM) gac410: Yes ...   winding down.    Thanks you everyone.  This was a great meeting
(10:35:03 AM) CDot: gac410: no, not given it much thought. jomo is on top if it, no?
(10:35:33 AM) MichaelDaum: gac410, thanks for bringing us all together regularly. much apprechated&required to keep going.
(10:35:46 AM) gac410: Well jomo has a suggested solution,   I wasn't sure you/he saw eye-to-eye on it.   I know he does NOT like the one I came up with ;)
(10:35:48 AM) jomo: i could make tests...
(10:36:29 AM) gac410: My fix is very narrow.  Fixes sorting only.   His would make foswiki work on osx   which is pretty importatnt 
(10:36:57 AM) gac410: jomo,  Tests woudl be good,  but you'd need to do the testing.  Not sure anyone else here has osx to test on.
(10:37:35 AM) MichaelDaum is now known as MichaelDaum_
(10:37:51 AM) gac410: Okay.   everyone.  Thanks.   I'll update the release plan and reset the timers on a couple of the proposals.   PLEASE provide feedack to the list of proposals in today's meeting.
(10:39:00 AM) Lavr: Thanks George
(10:39:03 AM) gac410: The window will close at our Next meeting 5 October.     Everything ready for  or merged by 1 December.    Please commit to Item branches if possible. 
(10:39:18 AM) JulianLevens: Thanks George
(10:39:53 AM) gac410: You're welcome.   And thanks for participating. We'll keep this moving along.   
(10:42:04 AM) andreli: Great work George, thanks a lot
(10:48:19 AM) jomo left the room.