Release Meeting: 4 Aug 2014

Details

1. Tasks review - tasks that need resolution prior to release

Continues from ReleaseMeeting01x02_20140721. Green shows current status

Tasks previously reviewed: Report any updates.

  • 12019 (Closed): JQuery fixes mixed in with some non-default extension work. MichaelDaum will resolve
  • 10484 (Closed): Search formfield issues, waiting on Sven & pharvey, comments about needing CDot input. CDot to investigate documenting the restrictions
  • 10203 (Closed): Mostly complete, Needs help with completing ajax MichaelDaum and gac410 will test registration changes
  • 11671 (Closed): Needs review, work maybe complete. Feature discussion.
    • Decided to mark waiting for release. The additional work depends on a feature proposal without a committed dev.
  • 12261 (Closed): Someone needs to update docs for the new PageCache.
  • 12477 (Closed): Spurious .changes entries. GeorgeClark will resolve
  • 12705 (Closed):Need help on this. CDot's patch resolves the corruption but causes   to be lost.
    • This is still urgent and needs help from CrawfordCurrie if at all possible.
  • 12855 (Closed): This can be closed any time. Left open to remind that we should do a final audit before building release.
  • 12925 (Closed): Page cache unit tests were all disabled in rewrite. StephanOsthold offered to help but has limited time over next few weeks.
  • 12926 (Closed): Subscribe links are broken. GeorgeClark will change SubscribePlugin rest handler to permit GET urls.
    • Some issues were encountered. MichaelDaum had some suggestions, George will revisit.
  • 12931 (Closed): CommentPlugin on trunk issues Need help with this one... CrawfordCurrie ?
  • 12958 (Closed): HTML label issue with editor. CrawfordCurrie proposed a patch. Can we apply this? Does it need unit tests?

  • 12965 (Closed): New item. This is an urgent cache corruption issue. MichaelDaum has requested help in resolving it. So far only FredTarbox has been able to recreate it. George agreed to test this once the git migration has made some progress.

2. Need some decision on how to resolve:

  • 12887 (No Action Required): Configure loop on Perl 5.8.8. JanKrueger update the Template parser, and on perl 5.8.8, a while loop over a regex gets "stuck" at the position of the first match. Needs to either be rewritten or someone with deep perl knowledge to fix the regex loop.
    • 12976 (Duplicate): is a duplicate. KennethLavrson tracked it down to perl Taint checking related. removing -T taint checking resolves both of these tasks. Need to decide if we can ship 1.2 with -T disabled for configure, or rewrite the code to resolve the issue.
    • JanKrueger will help work with GeorgeClark in the next week or two to try to resolve
    • GeorgeClark will create a feature proposal to remove -T taint checking from across foswiki. This resolves a number of issues but needs wider agreement.
  • 12050 (Closed): Another special case for squab links. Revisits DeprecateUndocumentedSqBracketLinkFormat Need to decide: Should we remove the deprecated link format from 1.2? See comments in task.
    • General consensus is we should remove this deprecated format. It's been officially unsupported for many years now.

3. Low hanging enhancements?

  • 10824 (Closed): Support for HEAD request, Should it be happen before cache processing?
    • GeorgeClark will post a revised patch for MichaelDaum to review.

4. git Migration

  • New branch tracking submodule repository has been created. https://github.com/foswiki/_allFoswiki
  • Git development flow documented in GitRepository
  • Decisions / issues to resolve:
    • Are we going to go with the superproject / submodule approach
      • Yes, but it's decided that superproject / submodule is optional. There is no particular reason that a developer has to choose to use submodules. JanKrueger suggested that http://gitslave.sf.net/ is a good alternative. The _allFoswiki will be kept around as a convenient starting point for new users. And as the root for building Foswiki releases.
    • Should we eliminate the _allDeveloper repo - this duplicates trunk as submodules and is probably not needed
      • Or maybe link it to all the popular / usable extensions, And download / use its .gitmodules file as an index of what pseudo-install should install. Currently it uses the subversion extension list. It could be converted to the github API, but the json request for all repositories is quite extensive - currently requires 21 fetch requests.
      • It's a good list of all the possible extensions in git, but the submodule foreach would be huge. Really not recommended to use this.
    • original https://github.com/foswiki/foswikirepositoryshould also be deleted
      • There are some branches not tracked elsewhere. Maybe needs some restructuring vs. removal.
      • This needs careful review. no definitive answer yet
    • What should we do with the https://www.gitorious.org/foswiki project? Once we turn off subversion, updates will stop.
      • Deferred. didn 't discuss.
    • Other github issues to resolve:
      • Discrepancies in number of repositories.
        • _allDeveloper links to 518 submodules
        • 610 repositories overall reported by "owner" group
        • FoswikCoreDevelopers group - 606 repositories (Excludes "foswiki" legacy repo)
          • There are 3 bogus repositories "data", "lib" and "test". These appear to be from a failed upload of the DownloadZipPlugin. They can probably be removed.
This is generally resolved. If we eliminate _allDeveloper, the rest has been resolved.
      • We need to go back and Tag all of our prior releases in the core and default extension repositories
        • Could be a manual effort, but it would be nice if a tool was available.
    • How will we allocate permissions:
      • Core developers vs. All developers
      • Additional groups related to extensions grouped by "CoordinateWithAuthor"
      • (Individual permissions per repository are not supported by github)
      • Not discussed - needs further review
    • The superproject _allFoswiki duplicates _coredistribution (I overlooked it)
      • _allFoswiki tracks the extensions by branch. However when an extension is updated, the SHA pointer in the superproject has to change. It's simple enough to a git submodule update --remote; git commit -a; git push. But do we want to track these maintenance updates by task? It will update each time a linked submodule is updated.
      • Not discussed
    • We need to get weblate running / hooked into github
      • Mentioned that this will happen once git migration completed, and Release01x02 is branched
    • Once we pull trigger, need to update the FoswikiOrgPlugin to post commits to the Tasks web.
    • Authors, committers, and matching github account to WikiName
      • The author is the original creator of a patch. If someone submits a "pull request" for modifications, they are the author. So the author may or may not have a WikiName on foswiki.org.
      • Then a core developer accepts the pull request, they become the committer. (at least I think that's how it works). They (hopefully) have a WikiName.
      • Both author and commiter are reported in the JSON transaction posted to foswiki.org. (Each with github username, name, and email address)
        • The Task will be updated using the identity of the Committer. So the github account has to map to a WikiName. We can either ask committers to use their WikiName as their reported github name. Or establish a mapping topic on foswiki.org.
      • We need to add the foswiki webhook to each repository, and catch up with all the current respositories.
        • URL: http://trunk.foswiki.org/bin/rest/FoswikiOrgPlugin/githubpush, Content type application/json

  • It was decided that we will set Subversion repository to READ ONLY on 8 August 2014, and complete the migration process after that

  • PaulHarvey agreed to help with an audit. One extension (FugueIconsContrib) is missing the most recent commit. We need to audit github and make sure subversion is completely in sync.

Other topics to review for background information:
  • Development/MoveCodeRepositoryToGit
  • Development/GitRepository
  • Development/GitConversionTasks
  • 11267 (Closed): (primary task)
  • 12819 (Closed): (Changes to Interwikis)

  • Next meeting - Monday August 18, 1300Z

IRC Log

(09:00:27 AM) gac410: Hi all. ... it's 9am - lets give it a few more minutes to see if anyone else joins.
(09:04:27 AM) MichaelDaum: hey guyys
(09:04:48 AM) gac410: hi MichaelDaum .. thin turnout so far. :(
(09:05:40 AM) ***gac410 wonders if we need to revisit the schedule for these meetings,
(09:06:09 AM) MichaelDaum: well people know we have RM now and here.
(09:06:28 AM) gac410: :)
(09:07:39 AM) gac410: Okay, lets get started. First quick review, any updates to report on previously reviewed / pending tasks.
(09:08:11 AM) gac410: I only had one significant one - Item12926 - broken subscribe links. Needs to be re-thought a bit.
(09:09:11 AM) ***MichaelDaum looks up what Item12926 is about
(09:09:32 AM) gac410: If we intend to allow GET, then the rest handler has to return an oops, instead of json, when invoked with GET.
(09:10:37 AM) MichaelDaum: canʼt say how Crawford implemented the error handler in SubscribePlugin
(09:10:55 AM) MichaelDaum: if it expecets a json in any case, then well, even the error message has to be embedded into a json
(09:11:19 AM) gac410: Two ways to trigger subscribe. A button w/ post. Works great. no issue there.
(09:11:53 AM) gac410: Or the old legacy url's in the top/bottom bar. Those are standard <a href= links. They are broken.
(09:12:35 AM) gac410: First, they violate the "only update from POST" guidelines. But if we relax that, then they also return json to the browser which is very confusing.
(09:13:30 AM) MichaelDaum: well it seems the js part is to blame. no problem to POST from js when a link gets clicked
(09:13:59 AM) MichaelDaum: as long as you don&#700;t need strikeone validation
(09:14:00 AM) gac410: Won't have a strikeone key. And since it doesn't start as a form, possibly none to get,
(09:14:21 AM) MichaelDaum: best is to allow unvalidated POSTs then
(09:14:27 AM) gac410: Ah. So another solution is to disable strikeone. Still needs js work though somewhat beyond me.
(09:15:23 AM) MichaelDaum: the js already does all the magic it seems
(09:15:34 AM) MichaelDaum: only problem is the rest registration requiring a validation
(09:15:36 AM) MichaelDaum: it seems
(09:15:59 AM) MichaelDaum: once that&#700;s done, no form is needed anymore
(09:16:14 AM) gac410: Okay. I'll experiment with it. I thought I ran into a No form found in dom error or something like that. Lets table and I'll try to dig into it next week sometime.
(09:16:47 AM) gac410: Any other updates for previously reported? everyone?
(09:16:49 AM) MichaelDaum: two changes: (1) use plain urls orchestrated by subscribe_plugin.js to issue a post (2) remove <form> markup otherwise
(09:17:03 AM) ModAcOst: Didn&#700;t have a lot of time unfortunately
(09:17:28 AM) MichaelDaum: me neither. had a lot of work on the plate the recent month so my tasks are all still pending.
(09:19:03 AM) MichaelDaum: what about Item11671?
(09:19:45 AM) MichaelDaum: spaceout not working as expected
(09:20:32 AM) gac410: The change to %SPACEOUT The fix is merged I think. But it runs into the "are numbers lower or upper case"
(09:20:51 AM) MichaelDaum: arg
(09:22:03 AM) MichaelDaum: well it seems we don&#700;t have anybody willing to work on it
(09:22:14 AM) gac410: Alexis committed the fix ages ago with unit tests, so my gut feel is that if nobody is complaining, just leave it as is.
(09:22:23 AM) MichaelDaum: ya
(09:22:41 AM) MichaelDaum: if there still is any issue left somebody could just file a new case
(09:22:59 AM) MichaelDaum: nothing is perfect
(09:23:12 AM) gac410: Okay, I'll change status to waiting for release. Maybe we can get someone to address the "Numbers as low or upper case" proposal .. that makes it configurable iirc.
(09:23:35 AM) MichaelDaum: k
(09:23:44 AM) MichaelDaum: http://foswiki.org/Tasks/Item12705 is a lot more critical
(09:25:03 AM) gac410: yeah. The fix is 99% there, but the disappearing nbsp's are a bear to resolve. I've tried twice and not gotten anywhere. It really needs CDot who deeply understands the Wysiwyg parser
(09:25:44 AM) MichaelDaum: I wasn&#700;t able to repro it frankly
(09:26:23 AM) MichaelDaum: for 1.1.9 deployments I still go with a lo-pro patch removing $_[0] = Encode::encode( encoding(), $_[0] );
(09:26:32 AM) MichaelDaum: to make wysiwyg not fail completely on 1.1.9
(09:26:34 AM) gac410: The unit tests fail for me. Maybe though because my system isn't on utf8.
(09:26:51 AM) gac410: Your quick fix though breaks dozens of unit tests.
(09:27:37 AM) gac410: And failures in the TranslatorTests scare me as they almost always indicate there will be corruption during the round trip.
(09:28:20 AM) MichaelDaum: it really requires CDot as he has got most insight into (1) unicode and (2) WysiwygPlugin
(09:28:53 AM) gac410: yes, I agree. Maybe now that he has ConfigureContrib checked in he'll be willing to review it.
(09:29:13 AM) MichaelDaum: cant say. he&#700;s not here.
(09:30:21 AM) gac410: Next time he's on I'll try to ping him on it. I'll change the task to waiting for feedback from him.
(09:30:29 AM) gac410: Anything else?
(09:30:49 AM) gac410: Lets move to #2. There are two issues 1) Item12887 / Item12976 Loop in perl due to regex bug. Lavr tracked it down to actual taint issue. jast wrote the looping code. Quick fix is to ship configure without -T flag
(09:31:40 AM) gac410: Ah. .. jast ...
(09:31:50 AM) gac410: 1) Item12887 / Item12976 Loop in perl due to regex bug. Lavr tracked it down to actual taint issue. jast wrote the looping code. Quick fix is to ship configure without -T flag
(09:32:58 AM) MichaelDaum: any insight why removing -T fixes the loop
(09:33:17 AM) MichaelDaum: a plain error in perl oldies?
(09:33:25 AM) gac410: It seems to be a perl bug. 5.8.8 and 5.10 Removing -T or newer perl fixes it.
(09:33:39 AM) MichaelDaum: ok oh well.
(09:33:53 AM) MichaelDaum: removing -T makes sense for other reasons too.
(09:33:59 AM) MichaelDaum: but that&#700;s on a different plate
(09:34:04 AM) gac410: I tried to change from [[:space:]] charaacter class matches to see if that would make a difference - remove locale influences - but no difference.
(09:34:27 AM) MichaelDaum: removing -T everywhere requiring a feature proposal and a bit of feedback from others
(09:34:57 AM) gac410: right. In theory we are beyond new features for 1.2, but that one does cure a lot of ills.
(09:35:30 AM) MichaelDaum: -T even sucks for local perls wrt the shebang line.
(09:36:14 AM) MichaelDaum: #!/usr/bin/env perl -T doesn&#700;t work
(09:36:27 AM) gac410: Ah.. that reminds me why I didn't normally use env. I knew there was some reason.
(09:36:36 AM) MichaelDaum: so I see at least 4 good reasons to drop -T everywhere
(09:36:48 AM) MichaelDaum: (1) locales
(09:36:51 AM) MichaelDaum: (2) performance
(09:36:52 AM) gac410: I just run perl -I ../lib rewriteshebang to deal with local perls.
(09:37:06 AM) gac410: Don't we also need the "w" flag to enable warnings?
(09:37:08 AM) MichaelDaum: (3) bug in old perls as reported in Item12887
(09:37:19 AM) MichaelDaum: (4) local perl friendly
(09:37:23 AM) jast: -w isn&#700;t exactly necessary
(09:37:37 AM) MichaelDaum: use warnings = -w
(09:38:13 AM) gac410: Anyone volunteer to write up the feature proposal then? use warnings is pretty much universal in our code. And I suppose we could even add that to the hook to check code
(09:38:17 AM) MichaelDaum: there are some scripts missing use warnings; while they already have user strict; ... so adding use warnings; everywhere lets you remove -w.
(09:38:57 AM) jast: I guess taint mode isn&#700;t seeing a lot of attention from perl core developers
(09:39:23 AM) gac410: Okay ... who is writing proposal to remove it?
(09:39:29 AM) gac410: :D
(09:39:34 AM) MichaelDaum: ... most don&#700;t care about the mixture of taint mode + web dev + localization
(09:40:27 AM) gac410: Silence is deafening :P
(09:40:45 AM) jast: I might be able to conjure up a workaround btw, if someone who has the right perl version is around (my dev wiki is way too customized right now for me to simply update to recent trunk, so I couldn&#700;t test it myself)
(09:41:04 AM) jast: there must be _some_ way to work around that
(09:41:53 AM) gac410: Jast ... I'll work with you. maybe in the next week or two. Even if we drop -T, would be nice to not loop. I crashed hard w/ OOM issues the first time I hit it.
(09:42:20 AM) gac410: And I guess I'll write up the proposal to remove -T Not exactly a big one.
(09:42:22 AM) jast: whoops :)
(09:42:45 AM) MichaelDaum: if not only for performance reasons: removing -T adds 10% according to Tim Bunce
(09:43:58 AM) gac410: Okay. Two more tasks to review. Item12050: proposes removing the old [[SomeTopic Some topic link text]] Space delimited links. It's been officially deprecated since cairo but was left in to avoid breaking really old content.
(09:44:05 AM) MichaelDaum: had a nice talk about performance profiling and optimizing code http://www.perl.org/about/whitepapers/perl-profiling.html
(09:45:00 AM) gac410: Item10824, Proposal to add support for HEAD request, which is used by some search appliances. Very easy fix, I'm just a bit unsure wrt/ positioning and the cache.
(09:45:09 AM) MichaelDaum: Item12050 +1 on removing long deprecated feature nobody really uses and which stands in the way to advance other issues
(09:45:56 AM) jast: I&#700;m in favour of removing really old syntax
(09:46:10 AM) MichaelDaum: gac410, let me check the patch on Item10824 ... sec
(09:47:14 AM) gac410: jast, yes me too. I'd like to get input from Kenneth (lavr) and SvenDowideit. They support some really old sites. Don't want to cause too much pain either. But they are not here.
(09:47:21 AM) MichaelDaum: the patch is a bit outdated, it doesn&#700;t apply anymore
(09:47:28 AM) jast: clear case of tough luck ;)
(09:48:00 AM) gac410: MichaelDaum: I did rework it, I think I have it stashed. I'll attach it and mark it waiting for your feedback.
(09:48:09 AM) MichaelDaum: gac410, ok
(09:48:21 AM) MichaelDaum: I&#700;ll check it against the page cache logic then
(09:48:44 AM) gac410: Okay. unless someone has any other task items, we can discuss the git migration.
(09:49:34 AM) gac410: #1 Superproject / submodule approach Do we continue with that path? MichaelDaum how has your experience been? Reasonable approach?
(09:50:00 AM) MichaelDaum: second. on open tasks. could somebody assist in reproducing http://foswiki.org/Tasks/Item12965 ?
(09:50:48 AM) MichaelDaum: I think this task requires to be made urgend again
(09:51:12 AM) gac410: I'm not really set up for mysql caching, and don't have any real volume of data on my systems.
(09:51:53 AM) MichaelDaum: data volume isn&#700;t an issue. Fred was able to repro a deadlock on an almost empty topic using jmeter
(09:51:55 AM) jast: and I have zero knowledge about the page cache
(09:52:35 AM) gac410: I think most of us are in that boat. MichaelDaum you've not been able to recreate at all? Could it be sensitive to sql engine version?
(09:52:44 AM) MichaelDaum: all it needs is: setting up jmeter and fire up a test with 10 threads reading the same page again. then a confirmation of a deadlock or none.
(09:53:39 AM) MichaelDaum: looking at the locks there seems to be some sort of a race condition deleting and inserting page cache meta data
(09:54:07 AM) MichaelDaum: I really wasn&#700;t able to repro it on sqlite, not even when using 50 or 100 threads
(09:54:43 AM) MichaelDaum: so yes. if there is an error then it only manifests itself in a time window opened when talking to a ream sql backend
(09:54:54 AM) MichaelDaum: ^ream^real
(09:55:35 AM) gac410: Maybe I can try. But I'd like to get the git stuff behind us. Too many active threads to manage :)
(09:55:51 AM) gac410: howdy pharvey1
(09:56:02 AM) pharvey1: hi
(09:56:13 AM) MichaelDaum: I will try as well. but I really need to make sure the error can be confirmed. the more independent setups we have the better.
(09:56:29 AM) MichaelDaum: okay then lets move to git.
(09:56:45 AM) gac410: Okay I'll add it to my list. Hopefully jmeter is not difficult to get running
(09:57:15 AM) MichaelDaum: thanks, gac410
(09:57:39 AM) gac410: Okay first ... has anyone other than MichaelDaum started using the submodule approach. Do we go ahead with the superproject / submodule concept.
(09:58:26 AM) gac410: I think it looks good. ... except for some comments I've read that claim it's "intended for links to projects where the target is slow moving"
(09:58:38 AM) pharvey1: depends on how much typing we like to do to switch branches. git submodule foreach checkout Release01x10 :)
(09:59:25 AM) MichaelDaum: my experience have been mixed when using the pre-build super projects.
(09:59:43 AM) MichaelDaum: as those aren&#700;t what I need on both ends of a scale
(09:59:58 AM) jast: another potentially useful mechanism is gitslave (third party tool)
(10:00:06 AM) MichaelDaum: _allFoswiki is not covering all plugins I have an eye on. _allDeveloper is too much.
(10:00:36 AM) MichaelDaum: so what I did was creating a git of my own and add foswiki submodules as required
(10:00:46 AM) pharvey1: I&#700;ve always used my own superproject, and added/removed stuff to/from it FWIW.
(10:00:47 AM) jast: http://gitslave.sf.net/
(10:01:03 AM) gac410: Right I think the _all* repo is really more intended for the newbie to get started with a quick core checkout.
(10:01:23 AM) MichaelDaum: or for release management
(10:01:51 AM) gac410: Then "git checkout -b MyCustomizedbranch" and you have a local branch you can add to easily.
(10:02:03 AM) jast: gitslave basically allows you to run the same git command on many repositories at once
(10:02:08 AM) MichaelDaum: jast, what does gitslave add that git doesnt have?
(10:02:30 AM) pharvey1: at work I tag both the superproject and the submodules when a release is cut.
(10:02:39 AM) jast: it doesn&#700;t use/require submodules
(10:03:27 AM) MichaelDaum: what does gitslave (not using submodules) add compared to git (using submodules)?
(10:04:20 AM) jast: you can commit/push/pull multiple repos at once
(10:04:25 AM) jast: (for example)
(10:04:56 AM) MichaelDaum: can&#700;t you do that using git submodule foreach &#700;cmd&#700; ?
(10:05:28 AM) pharvey1: reading the doc it seems to create a nicer experience than foreach
(10:05:45 AM) gac410: My concern with git submodules is that "git submodule update --remote" is really quite slow.
(10:06:40 AM) jast: slower than a normal pull?
(10:06:46 AM) gac410: So I don't see that gitslave and submodules are mutually exclusive are they?
(10:06:48 AM) jast: I haven&#700;t used --remote at all so far
(10:06:49 AM) pharvey1: slower than a big-fat-repo
(10:07:09 AM) MichaelDaum: ... you can&#700;t mix local and remote submodules under the same superproject
(10:07:12 AM) pharvey1: for that reason I don&#700;t recall every cloning every single module
(10:07:47 AM) jast: I believe gitslave doesn&#700;t impose in any way
(10:07:54 AM) gac410: If you don't use --remote, then the superproject gradually goes out of date. That's how you update the sha hash pointers in the superproject to point to the updated submodule
(10:08:11 AM) pharvey1: according to the doc it&#700;s like "gits checkout -b foo" instead of "git submodule foreach git checkout -b foo"
(10:08:20 AM) MichaelDaum: so what I was goign to experiment with is use the superprject for local checkins not being a submodule. while linking in any distant foswiki repo as a submodule.
(10:09:15 AM) jast: you can&#700;t commit stuff to the superproject that is in the scope of the subproject
(10:09:22 AM) jast: *submodule
(10:09:43 AM) gac410: re speed, I have no idea about relative speed v.s. normal pull. It just "felt" that the update --remote was taking a long time
(10:09:51 AM) jast: not sure if that&#700;s what you meant
(10:10:21 AM) MichaelDaum: jast, I was thinking of a normal git with stuff checked in ... plus some foswiki submodules in addition.
(10:10:48 AM) MichaelDaum: gac410, yes it is slow. felt very svnish.
(10:10:48 AM) jast: right... for your own personal superproject, I suppose?
(10:10:57 AM) MichaelDaum: jast, exactly
(10:10:58 AM) jast: speed depends a lot on how the server is doing
(10:11:19 AM) jast: in general, at least. no idea what --remote might be doing in addition to that.
(10:11:41 AM) jast: personally I don&#700;t update all the repos all the time
(10:12:29 AM) jast: hmm, looks like gitslave has its own thing similar to .gitmodules
(10:12:43 AM) gac410: For a "git intended" design. A superproject is where the bulk of the work was intended. The submodule is to point to a typically slow moving remote repo. Good example of original intentions would be TinyMCEPlugin, which would point to the TinyMCE project for the editor itself.
(10:13:02 AM) jast: yeah, it&#700;s ideal for that
(10:13:13 AM) jast: to the point you don&#700;t have to customize the submodule
(10:13:20 AM) gac410: right.
(10:13:35 AM) MichaelDaum: jast, problem I see with gitslave is not of technical nature rather than YATO (yet another tool) we depend on, that might go OOO (out of orbit) in 2 years or so .... and then we are stuffed.
(10:13:50 AM) jast: gitslave has been around for at least 4 years
(10:13:58 AM) jast: there&#700;s still the YATO factor, though
(10:14:05 AM) gac410: Though if you have to customize, you branch the submodule, and merge as desired.
(10:14:45 AM) gac410: So another question. If gitslave went away, would it be "inconvenient" or a disaster.
(10:15:11 AM) MichaelDaum: depends on the amount of our scripts involved at that time
(10:15:27 AM) MichaelDaum: we might face a pootle situation
(10:15:30 AM) gac410: I think that the git super/sub works well for managing the foswiki release process and a quick place for new dev to start.
(10:15:48 AM) jast: gitslave is just a helper
(10:15:48 AM) gac410: well pootle did NOT go away. But it does not integrate easily with git.
(10:16:03 AM) jast: much like if --remote got removed from &#700;git submodule&#700; we wouldn&#700;t be completed screwed, just inconvenienced
(10:16:12 AM) MichaelDaum: basically a dev can decide on its own whether to use gitslave or not as long as there are separate repos per extension.
(10:16:25 AM) gac410: Pootle died because the server update blew away the config and it's as much work to rebuild as it is to try weblate.
(10:17:14 AM) gac410: MichaelDaum: right. Based upon that I suggest we use superproject just to manage release process, it's there as a convenience, and expermient with / build some dev docs around gitslave.
(10:17:25 AM) MichaelDaum: a release manager makes up a submodules driven env; jast goes with gitslave; doesn&#700;t seem to be a problem as far as I see.
(10:18:11 AM) gac410: right. It's a personal dev decision, no impact on project. good. So #1, submodules with flexibility.
(10:18:24 AM) gac410: I'd like to just drop _allDeveloper. It's way too big.
(10:19:03 AM) MichaelDaum: wth why the _ underscore in the name?
(10:19:43 AM) gac410: Just to indicate that it's not normal project content? No idea. pharvey was it you or babar that came up with that naming convention?
(10:20:45 AM) gac410: And it's really late for pharvey1 ... so question for him. Will we break things if we push to github before svn has been changed to readonly?
(10:21:49 AM) gac410: For ex. For some reason FugueIconsContrib missed a svn commit. I did a git svn rebase. Is it safe to push, or do I need to do that on the foswiki.org server to keep things in sync?
(10:22:53 AM) gac410: Also, do you expect that we'll delete the foswiki/foswiki repo once svn is read only?
(10:23:24 AM) pharvey1: hmm
(10:23:42 AM) gac410: And question for everyone? What do you think about timing on this. When do we make subversion readonly.
(10:24:15 AM) pharvey1: Fugue missed the latest commit? Or some commit in the middle of the history was lost?
(10:24:23 AM) gac410: no the most recent.
(10:24:34 AM) gac410: It was one commit behind svn
(10:24:57 AM) pharvey1: perhaps our script got killed (checkload.sh or whatever it is?)
(10:25:22 AM) pharvey1: I&#700;d like to share a git subtree script which can re-create the bigfatrepo on demand
(10:25:37 AM) MichaelDaum: I am for switching to git asap.
(10:25:41 AM) pharvey1: I started it some weeks ago on the couch but then got distracted making a minimal docker-fied foswiki.
(10:25:47 AM) gac410: yeah. maybe. That brings up another point. Is there some easy way to determine if anything else is out of date before we move from svn.
(10:26:53 AM) pharvey1: I&#700;ll probably be staying home sick tomorrow. I&#700;ve got some shasum-checking scripts for a different project I&#700;ll try to throw at FW. I&#700;ll need to leave a svn up running before I sleep I guess :)
(10:27:13 AM) gac410: Maybe do a md5 sum on all the files in the svn tree against the github repo checkout
(10:27:37 AM) pharvey1: my previous strategy was a magic tar invocation on a whole dir and it takes the shasum of that
(10:27:50 AM) pharvey1: tar to stdout | shasum
(10:27:58 AM) gac410: That would work.
(10:28:23 AM) pharvey1: it&#700;s crappy performance but got me out of trouble a few months ago :D
(10:28:36 AM) jast: pretty complicated, though, you have to filter all the special files from svn and git, for instance
(10:28:42 AM) gac410: When does it make sense to turn off the svn hooks from pseudo-install.
(10:28:57 AM) jast: after switching to r/o, probably
(10:29:01 AM) gac410: Probably after any audit
(10:29:23 AM) gac410: If we had to spend a few days with svn readonly .. any big objections?
(10:30:22 AM) gac410: Probably a reason to keep the _allDeveloper around. Easy way to check out *every* submodule. Though it needs updating, only links to 518 modules v.s 610 in the account.
(10:30:59 AM) jast: easier to write a script that checks out all of them
(10:32:11 AM) gac410: Okay Today is Monday. 8/4 If we say that this Friday 8/8 and 1300Z, we turn subversion read-only. Would that timing work for you pharvey to help with an audit.
(10:33:10 AM) gac410: Any developers in the midst of something big that going r/o is going to be a hardship?
(10:35:42 AM) gac410: I think the current foswiki/foswiki "big fat repo" has stuff that is not tracked elsewhere. So we need to possibly restructure a bit before it gets deleted.
(10:35:45 AM) MichaelDaum: I&#700;ve moved all my pending checkins from my svn workarea to a new git workarea today
(10:35:54 AM) gac410: Excellent.
(10:36:39 AM) MichaelDaum: an email announcing the switch might be sufficient then
(10:36:42 AM) gac410: I think that if you've run pseudo-install, you should be able to do a "git svn dcommit" to push git commits to subversion.
(10:37:11 AM) MichaelDaum: there are still a lot of gotchas as far as I see
(10:37:15 AM) gac410: It's riskier to push to github because process on foswiki.org will auto push
(10:37:28 AM) MichaelDaum: for one: I don&#700;t want pseudo-install to botch with my super project&#700;s hooks
(10:39:02 AM) gac410: I'm not sure we need hooks at the superproject level ... at least the way we are using them. There is nothing in the superproject that we are tracking anyway.
(10:39:14 AM) MichaelDaum: right
(10:39:19 AM) MichaelDaum: yet still that happens
(10:40:49 AM) gac410: pseudo-install still needs some work. What it doesn't know .. when running from core, is the parent repo a real repo, or a superproject. Ie are you in foswiki/foswiki where the hook is really important, or in foswiki/_allFoswiki, where the hook doesn't matter.
(10:41:45 AM) gac410: Once we freeze svn and deprecate foswiki/foswiki then we can remove the hook checking from the project.
(10:41:53 AM) gac410: er. superproject.
(10:41:58 AM) MichaelDaum: ah ok
(10:42:42 AM) gac410: pharvey1: you still awake? Thoughts on timing of the freeze 8/8 1300Z followed by you helping with the audit?
(10:43:23 AM) pharvey1: ok
(10:43:57 AM) gac410: Another large effort. We need to apply the Release01x01x01 ... tags to the release managed repos. core + default I can do it by hand but a tool would be nice.
(10:44:11 AM) gac410: It's not urgent, it could be done anytime, but needs to be done
(10:44:50 AM) gac410: pharvey1: ok: Is that a yes, 8/8 works for me? :) Just to be certain.
(10:45:06 AM) pharvey1: 8/8 works :)
(10:45:26 AM) gac410: excellent. I'll get emails and annoucnement s out. Maybe add a banner to f.o as well.
(10:46:43 AM) gac410: One last questino. Mapping github account to wikiname. Probably use a Topic on foswiki.org to manage. May need to be a hidden topic,. in some cases what I get consistently is an email address.
(10:47:30 AM) gac410: With that, we are at an hour:45 Any other questions / comments / concerns, or do we "move to adjourn"
(10:49:15 AM) pharvey1: sounds good
(10:51:41 AM) gac410: Anyone else? Hello world?
(10:52:00 AM) andreli: adjourn is ok
(10:52:15 AM) andreli: thanks for your effort
(10:53:04 AM) gac410: Thank you all. Once we get the git migration complete, we can decide when to branch master -> Release01x02 and get started on the release process.

Topic revision: r7 - 04 Aug 2014, GeorgeClark
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy