Conversation with #foswiki-release at Mon 02 Nov 2015 08:00:10 AM EST on gac410@chat.freenode.net (irc)
(08:00:48 AM) gac410: Good morning. 1300z is early now .. Daylight savings ended in the US, ugh.
(08:18:53 AM) JulianLevens entered the room.
(08:37:33 AM) gac410: Hi all... Is there interest in a release meeting? I was contemplating another possible feature, didn't write it up yet.
(08:39:01 AM) gac410: add a "validate="regex" option to URLPARAM. So pages could validate/filter the input from the url rather than just grabbing it and hoping the filtering is enough
(08:42:31 AM) gac410: So you could do something like %URLPARAM{"effect" default="fade" validate="(show|fade|slide|blind)"}%'}" or something like that.
(08:44:14 AM) gac410: I'm getting closer on a PatchRelease01x01Contrib that will backport the major backwards compat issues to 1.1.x ALLOW = *, etc.
(08:45:05 AM) gac410: Also going to include all the perl deprecation fixes. so sites have a bail-out if they update their server and kill their wiki
(08:48:17 AM) Lynnwood [~Lynnwood@foswiki/developer/lynnwood] entered the room.
(08:56:32 AM) jomo entered the room.
(08:57:01 AM) jomo: hiall
(08:57:36 AM) gac410: hi jomo. ... so far no activity. I guess the daylight savings time change caught everyone.
(08:58:22 AM) jomo: :)
(08:58:46 AM) MichaelDaum: hi all
(08:59:03 AM) gac410: Hi MichaelDaum
(09:00:12 AM) MichaelDaum: been working on JQueryPlugin before leavign into vacations ... which I just returned from
(09:00:45 AM) gac410: Ah. I wondered ... you've been a bit scarce lately :) Hope you had a good vaca
(09:00:46 AM) MichaelDaum: some 3rd party libs received updates. more will probably follow.
(09:01:32 AM) gac410: I've been focused on the PatchRelease01x01Contrib Try to make it easier to install new extensions on older foswiki.
(09:01:34 AM) gac410: https://github.com/foswiki/distro/blob/Release01x01/PatchRelease01x01Contrib/data/System/PatchRelease01x01Contrib.txt
(09:01:35 AM) MichaelDaum: ya. but now I need the vacation from the vacation
(09:02:11 AM) gac410: Ya. catching up on email after a couple weeks away somewhat ruins the benefits ;)
(09:03:38 AM) MichaelDaum: got some important fixes for NatEdit as well: it does not run jquery.validate on invisible formfields ... which is the default in this lib
(09:05:13 AM) gac410: I figure a 2.0.3 could go out pretty much any time.
(09:05:20 AM) MichaelDaum: then ran a few nyt profiles to improve rcs contrib
(09:06:32 AM) MichaelDaum: and improved speed by a margin ... alas it disables noCheckinPending by default
(09:06:53 AM) MichaelDaum: as it is the culprit for lots of slowdown
(09:06:58 AM) gac410: minor tweaks to RcsStore ? ah... answered that. Won't fly for mainline I don't think.
(09:07:32 AM) MichaelDaum: y
(09:07:37 AM) gac410: We can't break foswiki for sites who edit files offline. Even foswiki.org counts on that.
(09:08:00 AM) gac410: We could maybe make that configurable?
(09:08:16 AM) MichaelDaum: what I've found out is that any %REVINFO{rev="n"}% will considerably slow down things even with n = max revs
(09:09:05 AM) MichaelDaum: NatSkin has got three calls to %REVINFO{rev="maxrev"}% all of which for an rcs to count the number of revisions.
(09:09:18 AM) MichaelDaum: ^all of which fork^
(09:09:31 AM) CDot [~crawford@foswiki/developer/cdot] entered the room.
(09:09:54 AM) gac410: CDot ... MD is discussing some RcsStore performance issues
(09:11:35 AM) MichaelDaum: the non-macro %REVISIONS% implemented in UI::View has got similar problems
(09:11:41 AM) ***CDot sees no discussion.....
(09:12:04 AM) CDot: ah, revisions. Yep, ongoing problem with RCS
(09:12:46 AM) MichaelDaum: there are some optimizations in Foswiki::Store ... in layers above the store impl that functions called _within_ the store impl itself wont profit from
(09:13:06 AM) MichaelDaum: such as ways to get the max num of revs
(09:13:27 AM) MichaelDaum: theres some reorg needed here
(09:14:52 AM) MichaelDaum: see_numRevisions() within and outside of the store impl
(09:15:42 AM) MichaelDaum: to fix %REVINFO{rev="maxrev"}% you'd need to know the num recs within the store layer to prevent the slow path, alas getting to know the max num revs is on a slow path itself as things are atm
(09:16:05 AM) MichaelDaum: ^recs^revs
(09:17:21 AM) MichaelDaum: so my current optimizations will potentially affect any store impl
(09:18:18 AM) CDot: A lot could be achieved by pucshing those optimisations down into the store impl itself. For example, a DB store has no need of embedded meta-data, but embedded metadata is essential for optimising a text-based store
(09:18:57 AM) gac410: I assume that this is 2.1 feature discussion and not a 2.0.3 patch discussion ???
(09:19:19 AM) MichaelDaum: these little info bits need to be available as easy as can be. even an sql query would be too much.
(09:19:38 AM) MichaelDaum: gac410, y
(09:20:35 AM) MichaelDaum: I started investigating as even on my new system with lots of cpus, ram and ssd foswiki did not feel as snappy as I was expecting.
(09:20:38 AM) CDot: MichaelDaum: hmmm. You are assuming that there will continue to be a "meta object" or equivalent in memory. A fast DB would be better serving *everything* using queries (though not necessairly SQL!)
(09:21:26 AM) MichaelDaum: yea well sort of
(09:21:41 AM) MichaelDaum: see REVINFO for example
(09:22:29 AM) MichaelDaum: it is okay to pay for the first call, but any additional expansion of REVINFO increases runtime linearly ... not good.
(09:23:16 AM) MichaelDaum: especially REVINFO with a rev param
(09:24:00 AM) MichaelDaum: to be precise we have got two problems here
(09:24:14 AM) MichaelDaum: (1) the first call to %REVINFO{rev="..."}% is too expensive
(09:24:20 AM) CDot: MichaelDaum: can you identify the choke points? Last time I looked (and it has been a while) it was a moving target. Some pages it's the store, some it's rendering. Some it's the sheer volume of crap sent to the browser.
(09:24:22 AM) MichaelDaum: (2) no caching
(09:25:33 AM) MichaelDaum: CDot, yes: getLatestRevisionID, _numRevisions, getRevision.
(09:26:09 AM) CDot: .... which are not an identifiable problem on DBStoreContrib.... did you look at PFS as well?
(09:26:22 AM) MichaelDaum: nope
(09:28:04 AM) CDot: ok. So, spending a lot of time firtling the core parts of the store is likely to be wasted.... though moving functionality into RCS and firtling it *there* is likely to bear fruit.
(09:28:48 AM) CDot: or, not wasted. The right phrase is "likely to f**k up other store implementations"
(09:30:43 AM) ***MichaelDaum not sure what firtling means ... my dict neither ;)
(09:31:06 AM) MichaelDaum: atm all optimizations I did are rcs contrib only
(09:34:14 AM) CDot: sorry, "firtling" means "poking around" or "hitting with a hammer until it works"
(09:34:59 AM) CDot: I'm not criticising, just observing that any core optimisations need to consider the other store impls.
(09:35:30 AM) CDot: I'd like to think we have pushed as much as is appropriate into the store impl, but there are still thinks in Meta.pm that make me uncomfortable.
(09:35:50 AM) CDot: .... starting with that horrendous hash
(09:39:57 AM) ***gac410 just created a new feature proposal. http://foswiki.org/Development/AddValidationsToURLPARAMMacro
(09:41:22 AM) gac410: Also added http://foswiki.org/Development/MakeItEasierToBlockSystemWebGuestAccess ... timer ticking for a 2.1 enhancement.
(09:42:10 AM) gac410: The latter could possible slip into a patch release as it would make it easier to update sites that have already locked down system web, and has no adverse impact that I can think of anyway
(09:43:34 AM) gac410: Other than as Michael pointed out - backwards compat with 1.1 which I'm working on a patch contrib for.
(09:45:21 AM) gac410: MichaelDaum: Can you put your performance recommendations into a feature proposal? ... maybe make some of the shortcuts configurable?
(09:46:11 AM) MichaelDaum: gac410, in general I am fine with MakeItEasierToBlockSystemWebGuestAccess however we need to be aware that we force both groups of users into an upgrade of any sort: (1) those that stay with pre 1.1.10 yet installing new plugins (2) those on post 2.0.2 installing old plugins that have System web topics
(09:46:38 AM) MichaelDaum: both will face problems
(09:47:00 AM) MichaelDaum: we need a kind of survey of plugins using System web topics for configuration
(09:47:09 AM) gac410: This proposal is NOT locking down the System web So old plugins on new systems would have no impact at all.
(09:47:34 AM) MichaelDaum: WikiGuests will
(09:47:46 AM) gac410: It is only adding the ALLOW rules to topics where a locked system web would break the wiki. And that needs a backwards compat change.
(09:47:57 AM) gac410: No. Michael, no impact if user does not change System.WebPreferences.
(09:48:07 AM) MichaelDaum: oh okay
(09:48:09 AM) MichaelDaum: well
(09:48:33 AM) gac410: I am NOT proposing changing the default access to system. Only make it easy, so if admin choses to lockdown, the wiki still works.
(09:48:40 AM) MichaelDaum: what I mean is: when wiki guest is denied view in System web, then any plugin using sttings in there needs an upgrade as well.
(09:49:13 AM) MichaelDaum: right? and when that does not happen the wiki will be broken
(09:49:20 AM) gac410: Yes. I don't know how we can address that for old / private plugins.
(09:49:48 AM) MichaelDaum: all we can do is list those plugins that need some love&care in that respect
(09:50:03 AM) gac410: What I'm proposing, is that Default, out-of-the-box, foswiki will continue to be fully functional if WebPreferences on System is set to deny guests. Right.
(09:51:26 AM) gac410: Also, if a user extracts Foswiki-upgrade-2.0.1.zip, it won't clobber the ALLOW settings required for a locked down System web.
(09:51:26 AM) gac410:
(09:52:21 AM) gac410: And based upon that, ... no changes to WebPreferences, it is probably safe to put this in a patch release rather than defer for 2.1
(09:53:10 AM) MichaelDaum: that I am not sure of: does the 1.1.x branch have a migration path at hand for ALLOW = * topics?
(09:53:47 AM) gac410: Yes. Not *quite* yet released. But I've got a PatchRelease01x01Contrib in the Release01x01 branch just about ready.
(09:53:50 AM) MichaelDaum: we really need to make sure people dont get into a catch22 situation
(09:54:12 AM) gac410: See https://github.com/foswiki/distro/blob/Release01x01/PatchRelease01x01Contrib/data/System/PatchRelease01x01Contrib.txt
(09:54:26 AM) MichaelDaum: okay.
(09:54:49 AM) MichaelDaum: as soon as we've got that in the wild will we be able to lock down System for guests....but not before ;)
(09:56:06 AM) gac410: I think I've got all the major backwards compat issues covered. I need to pick up a couple of your 1.1.10 proposed fixes though. The CLI and REST fixes probably are worth backporting to a pach.
(09:56:20 AM) gac410: patchContrib
(09:56:38 AM) MichaelDaum: xecellent
(09:57:55 AM) gac410: We really need to make a pass of our FeatureProposals, and either park the dead ones, or find someone active to pick them up. And then slot them into 2.1 or a future release.
(09:59:39 AM) gac410: http://foswiki.org/Development/ReleasePlan
(10:00:28 AM) gac410: 2.0.3 ... lets plan for a mid-November. And that's hopefully the final 2.0.x release
(10:01:33 AM) gac410: In the last release meeting, we decided Feature Freeze: 1 Dec 2015 - All proposed features tested and merged from Item* branches.
(10:02:00 AM) MichaelDaum: okay I make sure my jquery and natedit fixes are ready to go by then
(10:03:03 AM) gac410: For feature branches currently in github ...
(10:03:31 AM) MichaelDaum: could you update ReleasePlan, gac410? I'll then copy that into the FoswikiCalendar http://foswiki.org/Development/FoswikiCalendar
(10:03:40 AM) gac410: Okay. will do.
(10:04:01 AM) gac410: We have Item13594 branch. for the Param+= set implementation
(10:04:37 AM) gac410: Item13740 - add security headers
(10:05:20 AM) MichaelDaum: 2.1
(10:05:26 AM) gac410: Michael, your Item13525 branch with the Store cache ... is that defunct now?
(10:06:15 AM) MichaelDaum: I've got patches that should go in there extending this patch bit more
(10:07:17 AM) gac410: Do you still need another cache? I'm concerned that we might end up with too many caches. MetaCache StoreCache ...
(10:07:39 AM) gac410: JulianLevens: Is your Item13594 branch close to ready to merge?
(10:08:14 AM) MichaelDaum: gac410, yea right. current approach looks different.
(10:08:52 AM) MichaelDaum: it adds a {numRevs} hash to cache the max num revs of a topic.
(10:09:06 AM) MichaelDaum: as thats supprisingly expensive in rcs store
(10:09:23 AM) MichaelDaum: topic => int
(10:09:25 AM) gac410: Okay ... That seems pretty small. Vs. a full topic cache.
(10:09:33 AM) MichaelDaum: yep
(10:10:25 AM) gac410: So back to 2.0.3. ... I've got one task waiting for feedback from CDot ... but I think its good to go. We MUST encode # when in a urlparam. That's a possibly major breakage on 2.0.x
(10:11:00 AM) MichaelDaum: how is it potential breakage?
(10:11:01 AM) CDot: gac410: give me a link, I'll have a look now
(10:11:15 AM) gac410: http://foswiki.org/Tasks/Item13837
(10:11:35 AM) gac410: That is the ActionTracker bug. It really was broken. Param order was not a red herring :D
(10:11:53 AM) gac410: Any param with # would truncate the query string.
(10:11:57 AM) CDot: if it's ATP it can't possibly block a core release....
(10:12:05 AM) gac410: No.... It's a core bug
(10:12:08 AM) CDot: k
(10:12:27 AM) CDot: oh hell, yes, I guess it would
(10:12:30 AM) ***CDot reads on
(10:13:03 AM) gac410: A commit you did fixing unit tests for some reason removed # from urlEncode ...
(10:13:22 AM) gac410: But they all pass when I revert that change, so ... no idea what you were thinking there
(10:13:22 AM) CDot: I think # was added by default. I was probably reading a RFC that listed it.
(10:13:40 AM) CDot: me neiother. :-(
(10:14:03 AM) gac410: The RFC is confusing. It doesn't list # in the paragraph quoted. But elsewehere in text it says # must alsways be encoded.
(10:14:18 AM) gac410: Anyway... if fixored it.
(10:15:34 AM) gac410: Anyway, so with the http://foswiki.org/Tasks/PatchReleaseBlocker list. I don't see any reason not to just go with 2.0.3. I cannot figure out how to recreate the Extension installer issue.
(10:15:53 AM) gac410: Though it's *still* broken on foswiki.org. I cannot get MultiTopicSave to update the System topic.
(10:16:56 AM) gac410: There are enough tasks in 2.0.3, that I think we should get it out soon, http://foswiki.org/Tasks/TasksByRelease?release=2.0.3&checkins=All
(10:18:27 AM) gac410: So http://foswiki.org/Tasks/Item13785 is the only thing IMO blocking 2.0.3 ... and I'm at a loss.
(10:24:48 AM) gac410: MichaelDaum: You asked "how is that potential breakage" ... If A # appears in a query string un-encoded, it truncates the rest of the query string since it's the "fragment delimiter" aka Anchor
(10:25:02 AM) gac410: 2.0 no longer encodes # in urlparams
(10:26:12 AM) MichaelDaum: ah okay
(10:26:14 AM) MichaelDaum: sure
(10:26:24 AM) MichaelDaum: anything after some # is client side
(10:26:51 AM) gac410: It's been broken since 2.0.0 but only ActionTracker has tripped over it
(10:27:09 AM) gac410: Michael, how do I update that calendar again?
(10:27:50 AM) gac410: I updated ReleasePlan
(10:30:26 AM) MichaelDaum: see the info at http://foswiki.org/Development/FoswikiCalendar
(10:31:05 AM) MichaelDaum: but NM. I've got the calendar in my google calendar already. easy to add events to it.
(10:31:54 AM) gac410: Everyone. Please review http://foswiki.org/Development/ReleasePlan. If a feature is ready to merge, please set the CurrentState to ReadyToMerge.
(10:32:25 AM) gac410: If you don't think a feature will make the 1 Dec. feature freeze, please mark it for 2.2
(10:33:43 AM) gac410: Michale, all that page says is: Members of the Community.AssociationBoard have write access. I clicked edit, but there is no calender there to edit.
(10:33:53 AM) gac410: er. Michael (sorry)
(10:35:50 AM) ***MichaelDaum added the dates to the google calendar
(10:37:09 AM) gac410: Ah,... okay, and finally it shows up ... I had to use Chrome, not Firefox when I clicked the google calendar icon to get it iadded.
(10:38:19 AM) gac410: Anyway. EVERYONE ... homework. Please review the ReleasePlan and FeatureProposals. And make sure the 2.1 features are ready for the Dec. 1st feature freeze
(10:38:58 AM) MichaelDaum: Is Paul still interested in MakeZonesLessIntrusive?
(10:39:13 AM) gac410: I have not heard from him in a long time.
(10:39:50 AM) gac410: I suspect Sven, Paul, Arthur, features all need to be parked or otherwise put up for adoption,
(10:41:26 AM) jomo: if i read right - the patch is already included in the http://foswiki.org/Development/MakeZonesLessIntrusive - or it isn't ok?
(10:43:05 AM) gac410: Vladm is looking into changing Foswiki email to use Email::MIME so that the encoding for utf-8 can be better handled.
(10:43:28 AM) gac410: jomo, yes. I added myself to that particular one. even I can cut/paste :D
(10:44:43 AM) gac410: Ah yeah... So I guess unless someone else (michael?) picks it up, I'll get it ready.
(10:48:37 AM) gac410: I added a new FeatureProposal state. "ReadyToMerge"
(10:49:31 AM) gac410: So UnderConstruction says the Item/Feature branch is still being worked on. ReadyToMerge, ... safe to merge into master once it's ready to start accepting features.
(10:50:41 AM) MichaelDaum: okay
(10:51:22 AM) MichaelDaum: guys got a call comming in.
(10:55:02 AM) MichaelDaum is now known as MichaelDaum_
(11:06:22 AM) gac410: Let's wrap it up. Thanks everyone. See you in 2 weeks
(11:06:40 AM) jomo: cu :)
(12:09:59 PM) jomo left the room (quit: Quit: jomo).
(12:19:28 PM) MichaelDaum_ left the room (quit: Quit: quit).
(02:13:34 PM) CDot left the room (quit: Quit: Leaving.).
(03:51:46 PM) JulianLevens left the room (quit: Ping timeout: 272 seconds).
(06:20:12 PM) Lynnwood is now known as Lynnwood_
(07:15:13 PM) Lynnwood_ is now known as Lynnwood
(09:24:15 PM) Lynnwood left the room (quit: Quit: My Mac has gone to sleep. ZZZzzz…).