Release Meeting: 30 May 2016

(08:32:25 AM) JulianLevens  entered the room.
(08:51:04 AM) vrurg  entered the room.
(09:00:17 AM) gac410: Hi Everyone.  ...  
(09:00:24 AM) MichaelDaum: Hi George
(09:01:23 AM) gac410: Sorry for messing up the schedule and missing the last meeting.  
(09:01:46 AM) MichaelDaum: np, I was traveling anyway
(09:02:41 AM) gac410: First up usually is urgent tasks review.   There's only one PatchReleaseBlocker:  https://foswiki.org/Tasks/Item14067   
(09:03:04 AM) gac410: But as described that's not really a bugfix - a rewrite of the MetaCache
(09:03:09 AM) MichaelDaum: y
(09:03:16 AM) MichaelDaum: why is it scheduled for 2.1.3
(09:03:30 AM) gac410: No idea.   ;)
(09:04:03 AM) MichaelDaum: better we push that towards 2.2.x at least
(09:04:23 AM) gac410: I opened ... and closed one other blocker.   INCLUDE wasn't working correctly for "List,of,topics"   if a topic was not view authorized.
(09:04:31 AM) MichaelDaum: y
(09:04:34 AM) MichaelDaum: seen
(09:05:38 AM) gac410: Okay I pushed 14067 to 2.2
(09:05:47 AM) MichaelDaum: ah ok
(09:05:48 AM) MichaelDaum: thx
(09:06:19 AM) MichaelDaum: this one here should be released asap: https://foswiki.org/Tasks/Item14066#r5
(09:06:45 AM) MichaelDaum: https://foswiki.org/Tasks/Item14068 is already scheduled for 2.1.3 ... right?
(09:06:46 AM) gac410: Y and it's marked for 2.2, but fixed in 2.1.3
(09:07:00 AM) ***MichaelDaum fixing the report
(09:07:17 AM) gac410: yes   14068 is slotted for 2.1.3
(09:07:24 AM) MichaelDaum: okay
(09:08:15 AM) MichaelDaum: did you spot any other code that would benefit from the trick in 14066?
(09:09:05 AM) vrurg: Hi everyone!
(09:09:11 AM) gac410: I'm pretty sure there are other NFKD sorting.   TablePllugin,    and PlainFileStore too.
(09:09:19 AM) gac410: Hi vrurg
(09:10:53 AM) gac410: There are only 4 tasks fixed for 2.1.3,   so not to the level of generating a release yet.   Things have been really quiet
(09:11:28 AM) gac410: Which is okay,  I've been extremely busy with real life.
(09:11:47 AM) MichaelDaum: I've got one other performance fix that could go into 2.1.3
(09:12:28 AM) MichaelDaum: Foswiki::Templates::_readFile 
(09:12:38 AM) MichaelDaum: it is being called repeatedly for the same thing
(09:13:17 AM) JulianLevens: CleanUpFoswikiLocales would sort NFKD in a more locale correct way as well as perform better
(09:13:58 AM) MichaelDaum: any sorting should follow http://perltraining.com.au/tips/2004-12-17.html
(09:14:07 AM) MichaelDaum: no matter what the sort crit is
(09:14:24 AM) MichaelDaum: except for trivial ones
(09:14:53 AM) JulianLevens: Not in this case check out: Unicode::Collate::Locale
(09:15:06 AM) JulianLevens: I suspect it does that trick under the hood
(09:15:48 AM) MichaelDaum: ah it has got its own ->sort() method ... i.e. not using sort { yadda } @list
(09:16:22 AM) gac410: 2.1.2 was released the beginning of May,  so I don't see us releasing a 2.1.3 until August ...  unless something really urgent comes up.
(09:16:37 AM) JulianLevens: y, and it's core perl albeit from 5.13.4
(09:16:47 AM) MichaelDaum: awesome
(09:17:22 AM) JulianLevens: YARFTLP - yet another reason for the latest perl
(09:17:22 AM) gac410: So feature requests.   We were considering feature freeze for 2.2,  tomorrow.    Not much progress so I think we need to delay 2.2 .. probably to end of summer now?   September?
(09:17:51 AM) MichaelDaum: end of summer at least
(09:18:04 AM) MichaelDaum: we are all on vacation presumably ... more or less :/
(09:18:13 AM) JulianLevens: y, I commented on Backlinks proposal, I have to admit to having been effectively absent for a few months
(09:18:18 AM) gac410: I have the rewritten Foswiki::Request   ready to go,   I'll probably merge it to master soon.   Item14033 branch.    
(09:18:57 AM) gac410: JsonRpcContrib  uses Foswiki::Request::JSON   if it exists,  otherwise it falls back to it's own ::Request module.
(09:19:25 AM) MichaelDaum: ... which just answers the question I was going to ask
(09:19:42 AM) MichaelDaum: keeping JRC backwards compatible 
(09:19:57 AM) vrurg: gac410: Does your Item14033 branch passes the tests? Unit::Request doesn't seem to be compatible with your Foswiki::Request.
(09:20:09 AM) gac410: yes.  
(09:21:02 AM) gac410: I've mainly been focused on getting CacheTests working on 14033 branch.   I had to add Unit::Request::Rest and Unit::Request::JSON
(09:21:58 AM) gac410: As of my push last night,  14033 branches has the fixes from master Cache tests and also picks up vrurg's  find .. that "rest" cache tests were actually testing view.
(09:22:35 AM) vrurg: Ah, I see... Ok, I simply got rid of Unit::Request altogether but mine model is different.
(09:23:13 AM) gac410: So ...  given the dearth of features for 2.2 ... should we consider looking at 3.0 as our next big release and skip 2.2?
(09:24:04 AM) MichaelDaum: not yet imho
(09:24:13 AM) vrurg: Only if I'm getting backed now. Otherwise, considering my winter time experience, it'll take another 2-3 months to get all the tests working.
(09:24:23 AM) MichaelDaum: there are a couple if FPs that will role in soon that are worth a 2.2.x
(09:25:08 AM) gac410: We really need to work on merging master -> Item13897   so that the OO branch doesn't get too far out-of-sync with master.   Than that is going to be a bear of a job.
(09:26:10 AM) MichaelDaum: true but we also have enough problems in current master that would be overshadowed by the innate problems of moving to 00
(09:26:52 AM) MichaelDaum: I'd delay two things post 00: (1) rewrite of meta cache (2) rewrite of user code
(09:27:25 AM) gac410: My impression is that the new oo model will make some stuff much easier to find / fix.    The hoops needed to fix the Cache tests   look like they would be MUCH more straight forward with the new App initializer 
(09:27:54 AM) vrurg: Can we think of introducing universal caching framework?
(09:27:58 AM) JulianLevens: In a week or two I'll have time to be more active, no holiday plans in the immediate future
(09:28:24 AM) JulianLevens: What do you mean by UCF?
(09:28:55 AM) vrurg: Single point of caching everything. Perhaps based on CHI module as jomo proposed.
(09:29:08 AM) MichaelDaum: vrurg, thats the easy part
(09:29:26 AM) MichaelDaum: storing stuff isnt hard. getting rid of it at the right time is what is more of concern.
(09:30:16 AM) gac410: I'd say any significan features  ... anything that implies 'rewrite'   ought to be looked at for the 3.0 branch
(09:31:19 AM) gac410: Maybe we should rename Item13897 branch to be the "OOmaster" branch or something like that  to make clear our intentions.
(09:31:24 AM) vrurg: MichaelDaum: I didn't get much time to read the CHI docs but it looks pretty promising in every respect.
(09:32:03 AM) MichaelDaum: yea did probably the same amount of digging. looks nice. wanna have. yummy.
(09:32:54 AM) vrurg: But this is where gac410 comes with really good point: such major changes are to be destined for 3.0
(09:33:07 AM) JulianLevens: I have noted before a difference in ideas when talking caching
(09:33:29 AM) JulianLevens: For some like MichaelDaum he references the idea that entries are thrown away
(09:33:36 AM) gac410: This is really a major move.   The community needs to also get behind the recognition that just about every extension needs to be rewriten... And we need a new extension release mechanism
(09:33:45 AM) MichaelDaum: 3.0 is our ove to the new 00-model. any code changes on a larger scale should be postponed after that move has settled. imho.
(09:33:51 AM) JulianLevens: For other there is a sense of permanence like indexes in a DB
(09:34:18 AM) gac410: JulianLevens:  The CHI framework accommodates those concepts.   Caches have different lifetimes.
(09:34:25 AM) JulianLevens: Good
(09:34:54 AM) vrurg: JulianLevens: caching is speeding up access whily trying to maintain as low footprint as possible. This is my view. ;)
(09:35:01 AM) gac410: https://metacpan.org/pod/CHI
(09:35:19 AM) vrurg: Anyway, I propose skip the caching part for now. 
(09:35:55 AM) vrurg: This is the time where I do need help.
(09:36:22 AM) MichaelDaum: just one note on caching: entries do have a limited validity. just think of your backlinks proposal. we discussed storing outgoing links 4 years ago. these need to be cached somewhere ... and recomputed as needed.
(09:36:24 AM) gac410: So ... how do we help
(09:36:43 AM) vrurg: Pretty much the situation in my branch is described in two words: the model proved to be working. So, the sceleteon is there, we need to put some muscles into. 
(09:36:49 AM) gac410: See https://metacpan.org/pod/CHI#SYNOPSIS  ....   lots of different standard backends. 
(09:37:26 AM) JulianLevens: Love to help but I need a few weeks before I'm available, but your talking months of work right?
(09:37:40 AM) gac410: I'd like to tackle a merge from master ... to try to pick up all the changes.    It will be very painful effort.   
(09:38:13 AM) vrurg: In particular it means that as I'm still not pretty familiar with the logic behind some code it takes a lot of time to invetigade. As a result – exception handling is still undone. Some modules are still in their original pre-OO state. Etc.
(09:38:27 AM) MichaelDaum: just think of html page caching: a significant amount of work goes into dependency tracking to make sure no stale content is ever delivered. this is the flip coin of caching: purging with style.
(09:38:59 AM) vrurg: JulianLevens: 2-3 month to get all the bases tests passing. This wouldn't meant the code is ready for beta-test even.
(09:39:33 AM) JulianLevens: y, I get that idea, but I brought up the idea of a unified cache, in the DBCACHE sense, i.e. a key value API for generic use
(09:39:36 AM) gac410: So vrurg is clearly the one who understands the skeleton the best.   He now has lots of examples that we can work from.   and https://foswiki.org/Development/Foswiki3CodeChanges   is our beginning bible of how to transform.
(09:40:07 AM) JulianLevens: So, DBCACHE could support it by so could any Contrib providing SQL support
(09:40:55 AM) JulianLevens: But CHI seems to provide support for these backends
(09:41:12 AM) gac410: Some of the "simple" work to get feet wet might be to pick an extension,  and try to convert it.   And contribute to the Foswiki3CodeChanges as you go.
(09:41:36 AM) MichaelDaum: which brings us to rethinking the extension release process
(09:41:59 AM) MichaelDaum: we might end up having to maintain two flavors of the same extension
(09:42:07 AM) MichaelDaum: for a non-trivial time span
(09:42:10 AM) gac410: vrurg:    One area I started to dig into and got lost.    tools/configure     It does so much 'special' stuff related to config initializing.
(09:42:50 AM) vrurg: MichaelDaum: note that I want another plugins model in the future too. Perhaps a work for 4.0 but still a thing to think of.
(09:43:06 AM) gac410: MichaelDaum:   yes.   I suppose the low effort would be to create an Extensions3  web   or something like that.    Trying to implement per-extension versioning would be "nirvana" but a hell of a job.
(09:43:33 AM) JulianLevens: I've looked a lot at the Extensions and there's quite a lot needs fixing, including f.o. inconsistencies
(09:43:39 AM) MichaelDaum: is there a way to bundle one extension for both engines?
(09:44:02 AM) gac410: We've had numerous discussions and proposals over the years on how to version extensions,    never got done because it's a really hard problem.
(09:44:03 AM) vrurg: gac410: This is where I didn't look whatsoever. Foswiki::Config is completely different now, it took bootstrap code from the Configure – this is all config related staff I can tell you about.
(09:44:41 AM) JulianLevens: I've looked at Trying to implement per-extension versioning; and there are ways
(09:45:22 AM) JulianLevens: There are also choices of course; I suppose I'd better revisit that and write it up
(09:45:36 AM) gac410: MichaelDaum:  bundled extensions?  That woulid be a real challenge,  especially if we want to maintain the old unzip-and-go  model  where an installer is not needed.
(09:46:19 AM) JulianLevens: I've been looked bundled extensions; but that's somewhat further off
(09:46:20 AM) MichaelDaum: I mean the same zip. Is there a way to accommodate for both engines?
(09:46:56 AM) JulianLevens: By changing the install process, yes
(09:47:53 AM) MichaelDaum: I really cant say yet as I havent digged deep enough into the 00-branch
(09:48:09 AM) JulianLevens: Currently it just reads all Extensions on f.o. the query could add a current version and filter. The question is where to hold the diff versions
(09:48:20 AM) vrurg: The simplest versioning would be to have 3.x extensions in a subdir and learn the new code to look in that subdir in first place.
(09:48:40 AM) vrurg: Then the unzip-and-go might work.
(09:48:49 AM) JulianLevens: y, that's simplest and should work
(09:50:33 AM) gac410: hm    So the "unzip" model  unzips a   lib/Foswiki/Plugins/SomeExtension.pm      If that code is 2.x compat,  it breaks on 3.x.     What if our Unzip   unzipped a extension.zip and an extensions3x.zip    (nested zips)   
(09:51:52 AM) vrurg: gac410: could be a solution too.
(09:52:03 AM) gac410: The old model assumes  that extensions can unzip into the "foswiki" directory.   placing stuff directly in  pub, data, lib, bin, ...    Which is broken in several ways.  DB based stores, for ex.
(09:52:44 AM) MichaelDaum: ... which we dont have
(09:52:58 AM) vrurg: ... and won't have for some time
(09:54:13 AM) gac410: Anyway,  IMHO we need to move away from the unzip model.  should always use the APIs  to touch the store, etc.
(09:54:36 AM) vrurg: Anyway, some kind of per-package installer will be required over the time. But not until we find a volunteer to do it.
(09:54:56 AM) gac410: Easiest is just to fork the Extensions directory.
(09:55:29 AM) gac410: Avoids confusion on what's available for new OO version, etc.
(09:55:51 AM) vrurg: Except that the forking would have a side effect of having two copies of same extension per each installation.
(09:55:51 AM) gac410: But sticks proverbial head in sand regarding the underlying issues
(09:57:35 AM) gac410: vrurg:   We have 350+ some odd extensions.   There will be a period of time where OOfoswiki will have only a handful.   As they get "ported" over time.
(09:57:51 AM) vrurg: gac410: I hope to aim these issues in the new plugins code. But have no idea when would be able to get to it.
(09:58:32 AM) MichaelDaum: lots of open source projects faced this kind of change
(09:58:43 AM) JulianLevens: Let's not get bogged down by this, we can move forward and discuss this in detail - in and FP or Brainstorm on F.O
(09:59:05 AM) vrurg: BTW, perhaps somebody would manage to implement real compatibility layer which would allow the use of the old extensions wihtout rewrting. It seems to be possible though I don't want to spend time on it.
(09:59:12 AM) gac410: Y.   My closing comment is that there is
(09:59:40 AM) vrurg: Before we done I have another question.
(09:59:43 AM) MichaelDaum: seems so. lots of changes are search&replace.
(09:59:44 AM) vrurg: The last one. :)
(09:59:58 AM) gac410: ... sorry....   there is a real confusing situation with what extensions work with which releases.  Lots of "chafff" in our extensions web.     So a clear break makes some sense to me.
(10:00:06 AM) vrurg: Can I try incorporating Plack::*?
(10:00:45 AM) MichaelDaum: vrurg, the hell nobody will hold you back :)
(10:00:56 AM) gac410: What's the +/-  ?    There was certainly a lot of interest in a PSGI / Plack version
(10:00:59 AM) MichaelDaum: as long as we can separate issues and dependencies
(10:01:22 AM) vrurg: It's just my habbit of asking before adding more dependencies to the code.
(10:01:51 AM) vrurg: gac410: I already use PSGI 3-element return array as the internal standard.
(10:02:30 AM) vrurg: If I manage to implement PSGI engine then it may be possible to get rid of all other engines at once.
(10:02:30 AM) MichaelDaum: vrurg, before adding it as a core dependency try to introduce Plack:* independently of the core. 
(10:02:32 AM) gac410: y   That seemed to be jomo's hot button.
(10:02:49 AM) vrurg: MichaelDaum: how?
(10:03:01 AM) MichaelDaum: vrurg, by implementing a PlackEngineContrib or so
(10:03:54 AM) gac410: I thnk Jomo's point is that our engines are an adaptation layer,  which is totally redundant to PSGI's adaptation layer.    If we are fully embracing the latter, then we really should not need separate engines.
(10:04:19 AM) MichaelDaum: I know
(10:04:22 AM) JulianLevens: y, I agree
(10:04:27 AM) vrurg: MichaelDaum: I would simply add Engine::PSGI. It's too much to wrap it into a contrib and easy enough to remove/ingore if necessary.
(10:04:46 AM) MichaelDaum: ah ok, vrurg
(10:05:19 AM) MichaelDaum: the idea is to prevent plack symbols to propagate too deep into the rest of the core
(10:05:21 AM) vrurg: And if everything goes well 3.0 may come out really engineless.
(10:06:23 AM) vrurg: Plack won't be accessed directly anyway. Request is the only framework where it would be used if engines are gone.
(10:06:32 AM) gac410: The primary reason to add a separate Engine contrib is so that it can be "optional" wrt installation and dependencies,  installed on older foswiki, etc.     Now that all existing engines are added by default
(10:06:32 AM) gac410: I don't see a real need to add yet another engine contrib.    We might even consider just dropping some of the separate contribs.
(10:07:33 AM) gac410: Simplifying some of the many "contribs" and "plugins"  seems like a good thing.
(10:07:50 AM) vrurg: gac410: I just wanna do it step-by-step. Actual PSGI engine would be just a thin layer between Plack and request.
(10:08:13 AM) gac410: however it makes sense to you I'd say.    
(10:08:35 AM) gac410: So far your judgment on this stuff seems quite sound.
(10:08:42 AM) vrurg: Ok, I'll experiment with it.
(10:09:09 AM) vrurg: Hope to have 3.0 fully PSGI compliant.
(10:09:24 AM) gac410: That would be really great
(10:09:51 AM) gac410: So    to swing back the discussion a bit for review....
(10:10:09 AM) MichaelDaum: and finally nobody will write TWiki/Foswiki anymore
(10:10:19 AM) gac410:  -  Push 2.2 out to September code freeze?
(10:10:25 AM) MichaelDaum: +1
(10:10:51 AM) MichaelDaum: could you please add the timeline to the FoswikiCalendar?
(10:10:53 AM) JulianLevens: +1
(10:11:07 AM) vrurg: +1
(10:11:28 AM) gac410:  - Patch release in August  unless somethign really urgent  (like a security bug)  arises
(10:11:40 AM) MichaelDaum: +1
(10:11:43 AM) JulianLevens: +1
(10:11:48 AM) vrurg: +1
(10:13:07 AM) gac410: vrurg:   For devs coming online to your branch  would you prefer to treat Item13897 as a "master" under your control,  and people should branch/merge   for work?
(10:13:22 AM) gac410: Or can we just commit to 13897
(10:13:52 AM) gac410: I've been hesitant to tackle a merge from master because it might be disruptive to your in-process work.
(10:14:32 AM) vrurg: I didn't think about it before. But you're right, branching would be better.
(10:14:55 AM) vrurg: Unless we agree not to commit untested/unfinished code. 
(10:16:12 AM) vrurg: I would discuss it later outside of this chat.
(10:17:34 AM) vrurg: Anyway, time to go for me. Thanks everybody!
(10:17:34 AM) gac410: Okay.   I'll go with a branch to try to merge in master,   You can review it after O
(10:17:47 AM) gac410: I've gotten all the conflicts resolved
(10:17:54 AM) gac410: Thanks everyone.   
(10:18:21 AM) JulianLevens: Thank you
(10:18:30 AM) MichaelDaum: thanks
(10:18:35 AM) MichaelDaum: pretty exciting work
(10:18:45 AM) gac410: Okay  Calendar updated.   8/15 - patch release target.   9/1  2.2. feature freeze
(10:25:03 AM) JulianLevens left the room (quit: Ping timeout: 276 seconds).
(10:44:24 AM) jast: sorry for missing the meeting, I had another meeting at the same time
(10:45:12 AM) gac410: np ...  Anything to add?
(10:47:58 AM) jast: just got caught up on the log... I don't think I've got anything relevant