Release Meeting: 14 Dec 2015

(08:00:22 AM) jomo  entered the room.
(08:00:35 AM) gac410: Hi everyone ... goodmorning.
(08:00:41 AM) jomo: hiall
(08:04:54 AM) andreli: Hi, thanks for the ping.
(08:05:00 AM) CDot [~crawford@foswiki/developer/cdot] entered the room.
(08:05:18 AM) gac410: Sorry -- Didn't remember you were here, from across a screen change
(08:07:16 AM) gac410: So hi everone. I guess there are no further feature plans for 2.1. ... Do we make this our feature freeze for 2.1, and move toward bugfixes, for a Jan release?
(08:08:01 AM) gac410: If we freeze 2.1 now, then 2.2 will be the subsequent release, probably targeted for 6 months out ... july 2016
(08:09:40 AM) gac410: There is one feature proposal - http://foswiki.org/Development/AddTakeOutPutBackBlocksToFunc that has gathered a bit of discussion, but no objections.
(08:09:50 AM) gac410: Not for 2.1 though,
(08:11:27 AM) gac410: One other one could be 2.2 consideration, but CDot, you have an objection on it: http://foswiki.org/Development/AddFoswikiFuncWikifyWebTopicName
(08:12:20 AM) gac410: I think some of the objection has been mooted, the old link format with space delimiter is gone, removed from the code.
(08:12:21 AM) CDot: The putbackblocks - I share the same view as Julian
(08:14:09 AM) gac410: So don't add it?
(08:14:09 AM) CDot: on the other one, my concern was only that it wasn't described well enough.
(08:14:52 AM) JulianLevens entered the room.
(08:14:57 AM) CDot: mighgt as well add it, I can't see the API undergoing the sort of redesign Julian describes.
(08:15:36 AM) JulianLevens: What did I just miss?
(08:15:59 AM) gac410: Yeah it just seems if it's being commonly abused, adding a stub would make it easier for the stuff in render to actually be redesigned.
(08:16:13 AM) gac410: JulianLevens: Discussion of the take/out put/back blocks
(08:17:15 AM) JulianLevens: Thought so, and I'm not sure I'm really talking about a redesign of the API par se
(08:17:57 AM) JulianLevens: Just thought it might be better to have Foswiki::Func::Blocks with existing stuff where it is
(08:18:08 AM) CDot: JulianLevens: no, but you are right that adding takeout locks the API further into the current flow.
(08:19:11 AM) gac410: Well, imo these re really more of a utility than an API, ... If render remove them it is still useful to plugins to be able to put aside blocks for later processing.
(08:19:42 AM) CDot: true
(08:20:44 AM) JulianLevens: y, it's logical not physical
(08:20:56 AM) gac410: I thought about a utility/helper module vs. Func. But seemd like it was just making things more complex.
(08:21:57 AM) JulianLevens: Ultimately aren't we talking about exposing this to Plugin authors?
(08:22:21 AM) gac410: Yes. Well it's exposed already, just abused.
(08:23:15 AM) JulianLevens: Then it should be Func
(08:23:24 AM) JulianLevens: and exposed cleanly
(08:24:07 AM) gac410: There are a half dozen extensions that bust the API to access that function, and at least one that implements it's own version.
(08:24:57 AM) gac410: There was one other feature request I raised that has gotten no input, so just accepted "by default" ... but that makes me uneasy.
(08:25:03 AM) gac410: AddValidationsToURLPARAMMacro
(08:26:23 AM) gac410: Adding something like the Configure checkers, but to URLPARAM.
(08:26:47 AM) gac410: Anyway. no need to discuss that one here, just bring it up to get some visibility ;)
(08:27:39 AM) gac410: So the feature front has been quiet. Any further discussion on 2.1. Do we go ahead and get 2.1 out for January, and then open up for a 2.2 in July?
(08:28:19 AM) jast: makes sense to me
(08:28:30 AM) jast: I can't see anything exciting getting fully discussed *and* completed by january
(08:28:44 AM) gac410: Basically 7 features, plus some normalization work and bugfixes for 2.1.
(08:29:35 AM) JulianLevens: Back to takeout/putback; I'm not in anyway opposed to placing it in the API - it's already happening (abused) so formalizing would be better
(08:30:02 AM) gac410: That was my take. I think MichaelDaum agreed as well ... considering his +1 reply
(08:30:55 AM) JulianLevens: And Foswiki::Func::Blocks although I'd prefer that to stop Func.pm growing and growing over time. At least new stuff can be more discrete
(08:31:45 AM) jast: I don't really think that solves anything, it just pollutes a different namespace
(08:31:52 AM) jast: I'm all for separating stuff out, but not units as small as this
(08:32:07 AM) gac410: Okay. as of today we are feature freeze for 2.1. Bugfix only on master branch.
(08:32:20 AM) jast: that's exactly what one big namespace like Foswiki::Func is good for
(08:33:37 AM) JulianLevens: y, I realise it could go to the other extreme and be too fragmented
(08:33:47 AM) gac410: I'd like to hit a couple of bug discussions today. http://foswiki.org/Tasks/Item13880 I'm proposing a change to render I think it's a bug not a feature change.
(08:34:18 AM) JulianLevens: For now I'll drop this; but it does bug me right or wrong
(08:35:10 AM) gac410: We don't "protect" <meta .... /> or <link ... /> tags from being TML rendered. So thinks like <link class="blah WkiWord" ...> the WkiWord gets rendered into <a href...>
(08:36:28 AM) gac410: I can't think of any reason <meta> or <link> should get rendered inside the <...tag...> itself. And it is breaking ADDTOZONE ... Need to add <literal> or <noautolink> tags into the zone text, to stop the zone id from being rendered.
(08:37:20 AM) gac410: so ADDTOZONE id=WikiWord text="<link ... >" will break the link tag because the ID is inserted into the link class, and then rendered.
(08:38:16 AM) gac410: And one other bug... CDot .. I need your help for. It's a leftover unicode bug and I have been unable to find it.
(08:39:48 AM) jast: right now I can't see any reason against your proposed change
(08:40:17 AM) gac410: I opened http://foswiki.org/Tasks/Item13894 for it. If you attach a file and give it a non-ascii comment, the Meta in the topic is perfect, but the meta in the *attachment* version history is corrupted.
(08:41:21 AM) gac410: thanks jast, ya, I could not think of any case where you would render inside a tag. And it was causing breakage in real topics. due to missed noautolinks / literal tags.
(08:42:10 AM) JulianLevens: I'm with jast on this
(08:42:22 AM) jast: note that your proposed regexes will not do precisely the right thing if a tag is nested into the meta/link tag (as in the example that motivated the task item)
(08:43:32 AM) jast: or if any of the tag's attributes contain a literal '>'
(08:43:55 AM) jast: this is _probably_ not a big deal but some strange edge cases might slip through because of this
(08:44:04 AM) gac410: Is a bare > permitted inside a tag?
(08:44:17 AM) jast: the standard doesn't exactly encourage it, from what I recall, but browsers accept it
(08:44:31 AM) jast: only within attributes, of course, not elsewhere in the tag
(08:45:22 AM) jast: and, in fact, the markup that Lynnwood reported does just that
(08:46:15 AM) jast: the whole thing is highly questionable
(08:47:37 AM) gac410: jast, actually it does not nest tags. The noautolink and literal are not inside the meta, They are outside it.
(08:50:37 AM) gac410: ohhh I see, the <p> hm I know I tested this after my patch. and it is not broken.
(08:50:39 AM) jast: the <p class="foswikiGrayText"> is inside the <meta> tag
(08:50:40 AM) jomo: it is an pretty common to write <img title=">>> imporant <<<" src=....>
(08:50:48 AM) jast: no, in this specific case it should work
(08:50:59 AM) jast: it's just not guaranteed to work if there is anything problematic after the first nested '>'
(08:51:22 AM) gac410: yeah. our rendering code is certainly fragile.
(08:51:32 AM) jast: jomo: the correct way, according to spec, is to write title="&gt;&gt;&gt; and so on
(08:52:27 AM) jast: we should probably simply make no guarantees for nested '>'s, since they're not exactly ideal in the first place, and include both your fix and a fix for this markup Lynnwood found
(08:52:28 AM) gac410: The other markup broken by this was the new UserRegistrationParts
(08:55:36 AM) CDot: gac410: RcsWrap only, or RcsLite too?
(08:55:54 AM) gac410: it had an ADDTOZONE id=UserRegistrationStyle text="<link ... >" to reference a CSS stylesheet. UserRegistrationStyle was added to the link, and then rendered.
(08:56:12 AM) gac410: CDot: Both I think. It'd demonstratable broken on Foswiki.org.
(08:56:50 AM) CDot: you think, or you're sure? Are the characters incorrectly encoded in the history file?
(08:57:16 AM) CDot: it's one of these thinks that would be undebuggable without an example
(08:58:14 AM) gac410: I'm trying to find the test topics CDot.
(08:58:29 AM) gac410: I did not run the rlog from the cli to look at the contents
(08:58:45 AM) CDot: don't need to; text editor would be fine
(08:59:08 AM) gac410: Locally I tried both lite and wrap, and had corrupted comments in the attachment rev history
(09:00:26 AM) gac410: Here is an example from trunk http://trunk.foswiki.org/bin/attach/Sandbox/%C5%BDu%C5%BDu?filename=%C5%A0%C3%AD%C5%99%C3%AD%C5%A1.png;revInfo=1
(09:00:35 AM) gac410: Note the version history
(09:00:56 AM) gac410: well that didnt' work.
(09:01:26 AM) ***CDot has no idea what it should be, so can't comment
(09:02:45 AM) gac410: The comment should be " the ČáŘý file attached from OS X" but displays the ČáŘý file attached from OS X
(09:02:58 AM) jomo: click on the "manage" - and you will see the problem.
(09:03:51 AM) gac410: http://trunk.foswiki.org/Sandbox/%C5%BDu%C5%BDu
(09:05:06 AM) CDot: comments look fine to me....
(09:05:33 AM) jomo: click manage - scroll down to "version history"
(09:05:34 AM) gac410: Manage one of the atttachments, and view the "version history" comments
(09:07:15 AM) CDot: looking at the history (the ,v file) it's fine, so it's some double-decode in the UI, I suspect
(09:07:56 AM) gac410: Here is another one I just created: http://foswiki.org/bin/attach/Sandbox/TestItem13894?filename=%C5%A0%C3%AD%C5%99%C3%AD%C5%A1.png;revInfo=1
(09:08:13 AM) ***CDot suspects the template used to generate a version history
(09:08:28 AM) gac410: Why are those links not clickable. :(
(09:08:30 AM) gac410: http://foswiki.org/bin/attach/Sandbox/TestItem13894
(09:09:25 AM) gac410: Okay. back to the <link> and <meta> ... I'm going to merge them into master, and ask that everyone keep an eye out for possible issues.
(09:09:44 AM) gac410: I'll review the unit tests and see if I can add some ..
(09:10:49 AM) gac410: One last unicode thing. Jomo found the issue with why NFC normalization is so slow on some perls. The developer removed XS code to fix EBCDIC compatibility. p5p disagreed. Developer quit, p5p added XS back in.
(09:12:09 AM) gac410: Anyway. I'm trying to add back in NFC normalization .. but just in places where it breaks for OSX *clients*. to minimize overhead. And will then add it in but configurable for OSX servers on the file system side.
(09:13:06 AM) gac410: So plan is to ship master with NFC normalization at the network boundary. And make it optional at the file system boundary,
(09:13:19 AM) jomo: here are some risks - imagine a data/pub in NFS whrere the server is Linux but the NFS endpoint is OS X - and such scenarios...
(09:14:00 AM) gac410: And Jomo... I missed some NFC calls in CGI, so the issue you encountered on trunk might hopefully now be fixed.
(09:14:13 AM) gac410: trunk.foswiki.org that is.
(09:14:39 AM) ***jomo will check/pull ;)
(09:15:16 AM) gac410: jomo, even with NFS endopint, it's the file *names* that need to be NFC normalized, not the contents, correct?
(09:15:28 AM) jomo: now i finally installed freebsd into vmware on my os x notebook - so can do tests...
(09:15:40 AM) jomo: yes - filenames
(09:16:00 AM) gac410: that should not be as badly performing as the contents normalization.
(09:16:15 AM) gac410: smaller filenames than topics :)
(09:16:22 AM) jomo: when i finish with the tests - i will rewrite the normalisation Task...
(09:17:00 AM) jomo: imho - for now - we can live with the current - only one support question was about the OS X - yet. ;)
(09:17:34 AM) gac410: I know there are other OSX users with fw as server ... VickiBrown for one. Just has not encountered the issues I suppose.
(09:19:09 AM) gac410: andreli: Regarding your comment on Item13894 ... and /bin/rest/ImagePlugin/resize.
(09:19:32 AM) gac410: we should never utf-8 encode non-topic content. Sounds like a bug.
(09:20:44 AM) gac410: I've not tried that function, as ImagePlugin isn't a default extension. But it's installed on Foswiki.org. Could you possibly create a Task and a Testcase there?
(09:23:05 AM) andreli: I thought so. Shall I open a new task then?
(09:23:26 AM) gac410: please, yes, unless one is already open. That's one of MichalelDaums plugins
(09:23:30 AM) andreli: Ok, will do.
(09:24:09 AM) gac410: CDot ... I data dumpered the info hash received from store. And I can see the corrupted data coming from store
(09:24:25 AM) gac410: Gets the META comment correct, and REvision comment corrupted.
(09:25:42 AM) CDot: It's a binary file, so it has to be read binmode
(09:25:52 AM) CDot: probably the reason
(09:26:28 AM) CDot: no, scrub that, if it was just rcslite that might eplain it
(09:27:47 AM) gac410: Currently both trunk and f.o use RcsWrap, ...
(09:27:57 AM) gac410: But I did recreate it locally with RcsLite
(09:32:05 AM) CDot: well, it's correctly encoded in the ,v file, and the comment field is read the same way as every other field
(09:32:50 AM) gac410: I'm pretty sure in my "hacking" trying to track it down, I added an extra _decode() call and it fixed the rev comment and crashed with a wide print on the META comment.
(09:35:35 AM) gac410: Anyway, unless anyone has anything else for the release meeting, we can adjourn and move back to #foswiki.
(09:35:35 AM) gac410: We are in "feature freeze" for 2.1, bugfix only ... Please try to test trunk.foswiki.org and master branch as much as possible.
(09:35:54 AM) jomo: hm.. http://www.w3.org/TR/html5/single-page.html - cite from the 8.3 Serializing HTML fragments: "If the algorithm was not invoked in the attribute mode, replace any occurrences of the "<" character by the string "&lt;", and any occurrences of the ">" character by the string "&gt;". - e.g. in the HTML5 exactly the ">" and "<" in the attributes like title=">>>some<<<" are NOT escaped... but maybe - i'm understand it wrong...
(09:37:00 AM) gac410: I'll target a year-end Beta, and a mid-January 2.1 release
(09:37:05 AM) CDot: gac410: where did you add the "Dump"?
(09:38:04 AM) gac410: tbh, it was after midnight last night,and I added them all over the place. ... kept working my way deeper from meta to store. And didn't stash my code
(09:38:29 AM) CDot: AFAICT the comment is handled in exactly the same was as the author, so if the comment is banjaxed, the author must be as well
(09:38:39 AM) ***CDot is doubtful that the store is at fault
(09:39:08 AM) gac410: Here is a dump from Store getInfo routine
(09:39:19 AM) gac410: getInfo: $VAR1 = \{
(09:39:19 AM) gac410:            'date' => 1450068952,
(09:39:19 AM) gac410:            'version' => 2,
(09:39:19 AM) gac410:            'author' => 'BaseUserMapping_333',
(09:39:19 AM) gac410:            'comment' => 'Attach file to Ã<85>½uÃ<85>½u with unicode comment - RcsLite'
(09:39:19 AM) gac410:          };
(09:39:41 AM) Lynnwood [~Lynnwood@foswiki/developer/lynnwood] entered the room.
(09:40:18 AM) CDot: gac410: wrap or lite? The implementation of getInfo is different for each
(09:40:26 AM) gac410: That one was lite.
(09:40:47 AM) CDot: and what makes you think it is corrupt?
(09:41:26 AM) CDot: or are you opening STDERR :utf8?
(09:42:03 AM) ***gac410 is clueless. Just adding print STDERR Data::Dumper
(09:42:20 AM) gac410: and from another dumper: comment' => "Attach file to \x{17d}u\x{17d}u with unicode comment - RcsLite"
(09:43:05 AM) CDot: well, I'm puzzled. I copied the example down locally, but I don't see the comment at all in the version history
(09:43:23 AM) CDot: so there is some subtle difference in the configuration that I'm missing
(09:43:25 AM) gac410: And at another location: 'comment' => 'Attach file to ŽuŽu with unicode comment - RcsLite'
(09:44:49 AM) gac410: I just recreated it a few minutes ago here: http://foswiki.org/bin/attach/Sandbox/TestItem13894
(09:44:52 AM) CDot: I'm pretty sure it's to do with the fact that attachments are binary, but beyond that.....
(09:45:00 AM) gac410: Click manage attachments to see.
(09:45:13 AM) CDot: recreating it on f.o is pretty pointless. Can't debug it there.
(09:45:24 AM) CDot: Need to recreate it locally, and right now I can't.
(09:45:50 AM) gac410: Simple. Create topic. Upload attachment, with a utf8 comment, then manage attachments
(09:46:17 AM) gac410: Are you sure you are using rcs and not plainfile?
(09:46:20 AM) CDot: d'oh - the subtle difference? I'm using PFS ;-/
(09:47:28 AM) CDot: ok, reproducted it. Shound be able to degrub it now.
(09:59:08 AM) jomo: gac410: thinking about the NFC. To ensure homogenity, we must: NFC every filename, and readdir result.
(09:59:15 AM) jomo: But probably is enough NFC the topic content ON THE SAVE.
(09:59:27 AM) jomo: E.g. if foswiki could ensure than every topic is saved as NFC, not need NFC on every read...
(09:59:46 AM) jomo: (but the question is still opened what about the Extensions - e.g. when extension returns some content...)
(10:00:28 AM) jomo: the save is occasional - in compare with reads...
(10:00:34 AM) gac410: If we always NFC on input from the network, why would we need to also NFC in save. Save is utf8, not nfc unicode.
(10:01:18 AM) gac410: so you are saying save should do encode_utf8(NFC(thetext)) ??
(10:02:52 AM) gac410: Before I make more changes could you verify correct OSX client operation on http://trunk.foswiki.org/ ??? I should be NFC encoding all of the network input now.
(10:03:05 AM) jomo: because here is difference betwwen:: perl -MUnicode::Normalize -Mutf8 -MEncode -E 'say Encode::encode_utf8(NFC(q{á}))' | od -bc
(10:03:13 AM) jomo: and perl -MUnicode::Normalize -Mutf8 -MEncode -E 'say Encode::encode_utf8(NFD(q{á}))' | od -bc
(10:03:43 AM) jomo: note - the "encode" - e.g. output...
(10:04:43 AM) gac410: I understand that. I'm asking if all input *from the network* is NFC(), then why do we also need to NFC() on save to topic?
(10:04:48 AM) jomo: i will test it as i said - now just "thinking" loud..
(10:05:51 AM) jomo: i'm not sure...
(10:07:12 AM) gac410: I want to keep the Network side separate from the file system side. Let's confirm (as close to) 100% correct *client* operation first. put that one to bed. Then move on to deal with file system concerns.
(10:07:19 AM) jomo: to many differnet inputs - network, filesystem, extensions (which generates topic content and/or attachments) - like AttachContentPlugin, XLS transformations - simply - better sleeping when have homogenity... :)
(10:07:26 AM) Lynnwood left the room (quit: Ping timeout: 240 seconds).
(10:07:51 AM) jomo: and you're right - the NFC on very read could be eliminated...
(10:07:59 AM) jomo: (i hope) :)
(10:08:26 AM) jomo: mean read the topic content - not filesystem
(10:08:27 AM) gac410: Right. AttachContentPlugin, TopicInteractionPlugin, etc. have to deal with NFC as well if they access the raw query string. I *think* if I got the CGI changes right, param() etc. should be NFC'd
(10:09:21 AM) ***gac410 going back to #foswiki. Let's close the release meeting. So I don't have too much to wrangle with the meeting logs.
(10:09:28 AM) jomo: okay - just thinking loud :) - will does test...
(10:09:34 AM) jomo: Have a question.
(10:10:24 AM) jomo: IMHO going to PSGI mean openeing MANY (every) cans full worms or even full of dragons - need discuss. So, should i create some topics with questions for the "problematic" themes, or here ISN'T a need to talk YET about them? (And no, imho this isn't a category - "just fork and start hacking" - too many impacts. Anyway - the "questions" ARE NOT an formal proposals.
(10:10:44 AM) gac410: Next meeting. Let's skip December 28. So next meeting is 1300Z on Monday Jan 11, 2016
(10:10:52 AM) jomo: Where i should write my *questions* on which we need agree and get an resolution? Into the Development.MakeFoswikiPSGIConformant? or create one "gateway" topic for the PSGI with links to other smaller topics, one for each decison?
(10:10:53 AM) gac410: jomo, make it a Brainstorming topic.
(10:11:12 AM) jomo: ah - ok... :)
(10:11:20 AM) gac410: I don't think yhou need a toipic per decision. That seems like overkill
(10:11:24 AM) jast: unrelated side note, I added some definitions to the unified auth topic
(10:11:39 AM) gac410: Yes I saw that jast. Thank you. Good clarification.
(10:11:41 AM) jast: they're not final, so feel free to comment or something
(10:12:31 AM) jomo: ok - thats all from me ;)
(10:13:31 AM) gac410: jomo, so if PSGI also takes over authentication, may want to comment on the evolving "requirements" discussion in http://foswiki.org/Development/UserAuthMapping2dot0
(10:14:53 AM) gac410: So... everyone get that... Next meeting 1300Z on Monday Jan 11, 2016.... Merry Christmas / Happy New Year / Good Holiday of your Belief ...
(10:15:19 AM) jomo: PSGI touches EVERY aspect of the FW .. ;( Authentication is one of them.... but the topic is currently "Not technological" - and i'm unable to thing in some astral-planes... ;) so, not commenting it .. *YET, until we start talking about the technology)
(10:16:01 AM) gac410: And for new years resolutions. .... anyone care to give me a break on Foswiki 2.2 ... and be the Release Manager. It's a very rewarding position, though the pay sucks.
(10:16:25 AM) jomo: no meeing in christmas? :) okay then :) cu in jan :)
(10:17:01 AM) gac410: If anyone wants to meet fine by me, but I'll be on the road. Have a 10 hour drive
(10:17:16 AM) jomo: hmm... imho you must quit foswiki - if want quit the RM role .. ;)
(10:17:27 AM) gac410: :P
(10:18:59 AM) gac410: Ohh Ohh... Everyone. Somewhere along the way, core got very untidy. I need to run a perltidy at some point. Now that most of the open feature branches have been merged is probably a good time.
(10:20:14 AM) jomo: imho Lavr will strongly disagree... ;)
(10:22:17 AM) jomo left the room.
(10:24:03 AM) jast: I'd be okay with doing RM, I guess
(10:24:18 AM) jast: I'm not in the association or anything, though
(10:24:59 AM) gac410: I don't think RM needs to be in the association.
(10:25:29 AM) jast: is the position spec'd somewhere?
(10:25:51 AM) jast: just so I know whether the pay is worth it ;)
(10:26:55 AM) gac410: Hm not that I know of. Running the release meetings, Building the beta's and releases, wrangling the ReleaseNotes, Slapping hands for slipping features into patches,
(10:27:16 AM) gac410: Announcing the releases, (team effort)
(10:27:26 AM) gac410: uploading
(10:27:34 AM) jast: slapping hands sounds good
(10:28:28 AM) gac410: If you want, I'll try to write up what I've been doing over the past releases, and will make it a "JobDescription" ... in the Development web. You can think it over and we can decide once 2.1 is out.
(10:29:58 AM) gac410: The worst part is probably getting reasonable release notes from the mess-that-is-task-summaries, Verifying that all checkins have a task that is appropriate for the release notes. ... closing them all. Uploading extensions, etc. Bleh.
(10:36:16 AM) andreli left the room (quit: Remote host closed the connection).
(12:54:06 PM) JulianLevens left the room (quit: Ping timeout: 240 seconds).
(01:22:29 PM) CDot left the room (quit: Quit: Leaving.).
(01:00:27 AM) The account has disconnected and you are no longer in this chat. You will automatically rejoin the chat when the account reconnects.