Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* kaarefc joins01:20
* kaarefc leaves02:07
* kaarefc joins02:40
* beatyfull joins03:34
* beatyfull leaves
* github-ff joins06:32
[fcrepo4] fasseg force-pushed fixity-projections from 31a45fa to 9003ddf: http://git.io/TnLOOg
fcrepo4/fixity-projections b1e60c6 fasseg: added projection mixin type and a FileSystemConnector which is able to inject this mixin type
fcrepo4/fixity-projections cc52634 fasseg: added CacheEntry interface and implementation for projected entries
fcrepo4/fixity-projections 92615b7 fasseg: updated creation of fixity results so the new CacheEntry interface is used
* github-ff leaves
* github-ff joins06:37
[fcrepo4] fasseg opened pull request #195: Update of fixity infrastructure in order to be able to create Fixity results for projected Datastreams (master...fixity-projections) http://git.io/rd8anA
* github-ff leaves
<pivotal-bot>Frank Asseg added comment: "Added PR https://github.com/futures/fcrepo4/pull/195" https://www.pivotaltracker.com/story/show/5600291606:39
Frank Asseg finished "fcr:fixity bug using the FileSystemConnector" https://www.pivotaltracker.com/story/show/56002916
* travis-ci joins06:46
[travis-ci] futures/fcrepo4#1366 (fixity-projections - 9003ddf : fasseg): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/31a45facb25d...9003ddf34736
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/15163187
* travis-ci leaves
<bljenkins>Yippie, build fixed!07:00
Project fcrepo4 build #1508: FIXED in 28 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1508/
Project fcrepo-fixity-corrupter build #527: SUCCESS in 1 min 36 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/527/07:01
Project fcrepo-kitchen-sink build #698: SUCCESS in 5 min 33 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/698/07:07
* github-ff joins08:44
[fcrepo-jms-indexer-pluggable] yecao opened pull request #18: Solr indexer improve (master...solrIndexerImprove) http://git.io/W2vtjQ
* github-ff leaves
* awoods joins09:15
* kaarefc leaves09:17
<pivotal-bot>Andrew Woods added comment: "Yes. The idea is to do something similar to: https://wiki.duraspace.org/display/FF/SPARQL+Recipes" https://www.pivotaltracker.com/story/show/5745338809:27
* kaarefc joins09:32
* ksclarke joins09:47
* escowles joins09:54
* kaarefc leaves10:01
* kaarefc joins
* jonathangee leaves
* mikeAtUVa joins
* jonathangee joins10:07
* ajs6f joins10:10
awoods: PIng?
* ermadmix joins
<pivotal-bot>A. "Horbulaco" Soroka added comment: "If that's what's wanted, it needs to come after an as-yet hypothetical ticket to generate some bas..." https://www.pivotaltracker.com/story/show/5745338810:13
* kaarefc leaves
<ajs6f>ermadmix: With awoods out, can you take some time today to a) review my indexer PR and b) figure out what Ye sent us?10:18
(with me and anyone else who can help, that is)
* osmandin joins10:19
<ermadmix>ajs6f: sure, beginning to look at it now
<ajs6f>ermadmix++10:20
* gregjansen joins
<ajs6f>I'll start looking at Ye's PR.
* github-ff joins10:24
[fcrepo4] gregjan opened pull request #196: adding 2 role authorization projects to fedora base project (master...master) http://git.io/x4aTiQ
* github-ff leaves
<pivotal-bot>Gregory Jansen added comment: "https://github.com/futures/fcrepo4/pull/196 pulling to fcrepo" https://www.pivotaltracker.com/story/show/6201958610:25
Gregory Jansen added comment: "https://github.com/futures/fcrepo4/pull/196 pulling to fcrepo" https://www.pivotaltracker.com/story/show/62019734
Gregory Jansen started "move fcrepo-auth-roles-basic into fcrepo" https://www.pivotaltracker.com/story/show/62019734
Gregory Jansen started "Add access roles API to fcrepo-webapp" https://www.pivotaltracker.com/story/show/6201990210:27
* ajs6f leaves10:29
* ajs6f joins10:31
ermadmix: As a side note, I realized this morning that IndexableContentRetriever extends Callable<InputStream> should extends Future<T> instead, with T varying according to the kind of retrieval occurring (a Map<String, Object> for named fields, an RdfStream for RDF, etc.), to avoid the extra parsing going on in there. I'm not going to push that now, so we can get caught up with these PRs, but I wanted you to know that I'm aware of it, because you'll no
ed it soon (if you haven't already).10:32
* tecoripa joins
* tecoripa leaves10:33
* fcrepo-bot joins10:40
<pivotal-bot>A. "Horbulaco" Soroka added "Write tickets for improving test coverage on indexer gear." https://www.pivotaltracker.com/story/show/6215617410:43
<ajs6f>afk bb for standup10:49
* ajs6f leaves
* ajs6f joins10:53
<mikeAtUVa>standup https://plus.google.com/hangouts/_/calendar/eW91cm1lZGlhc2hlbGYuY29tXzVlYzdpNXQ2Z282dTdidHI4aTVrbGJxOTUwQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20.opn3ai06k1ge0no877ei3cku4o11:01
<ajs6f>awoods: I though you were out?
* barmintor joins11:02
<awoods>ajs6f: I am11:03
<ajs6f>awoods: Then I'll ignore everything you say.
<awoods>barmintor/cbeer standup11:04
<barmintor>sorry I'm late- is there a standup?11:05
<ajs6f>There is now.
Although I'm sitting down.
<barmintor>can you link it please?
<ajs6f>https://plus.google.com/hangouts/_/calendar/eW91cm1lZGlhc2hlbGYuY29tXzVlYzdpNXQ2Z282dTdidHI4aTVrbGJxOTUwQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20.opn3ai06k1ge0no877ei3cku4o
<mikeAtUVa>standup https://plus.google.com/hangouts/_/calendar/eW91cm1lZGlhc2hlbGYuY29tXzVlYzdpNXQ2Z282dTdidHI4aTVrbGJxOTUwQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20.opn3ai06k1ge0no877ei3cku4o
<cbeer>mikeAtUVa: it's not letting me in.11:06
<ajs6f>http://www.youtube.com/watch?v=IOxpPJYUTDM
<mikeAtUVa>cbeer: glad you could finally get in...
<cbeer>took forever.
<pivotal-bot>Gregory Jansen finished "move fcrepo-auth-roles-basic into fcrepo" https://www.pivotaltracker.com/story/show/6201973411:11
Gregory Jansen finished "move fcrepo-auth-roles-common into fcrepo" https://www.pivotaltracker.com/story/show/62019586
Gregory Jansen edited "Expose AuthZ Service/API on Object/Datastream views in F4 webapp" https://www.pivotaltracker.com/story/show/6185048211:12
<barmintor>awoods, escowles: But some cloud storage is not asynchronous11:17
<pivotal-bot>A. "Horbulaco" Soroka added "Upgrade HTTp client in use in indexer" https://www.pivotaltracker.com/story/show/6216066011:21
A. "Horbulaco" Soroka edited "Upgrade HTTP client in use in indexer" https://www.pivotaltracker.com/story/show/62160660
<cbeer>ajs6f: i wonder if we were intentionally not scanning fcrepo-transform in -webapp.
<ajs6f>ermadmix: {blushes} How sweet!
<cbeer>ajs6f: kinda seems like we've thrown the whole kitchen sink into -webapp now
<ajs6f>cbeer: Why wouldn't we?
cbeer: Well, yeah. I feel that way too.
But without a modularization apparatus...11:22
ISH.
<osmandin>awoods: I updated the F3 value on escowles page : https://wiki.duraspace.org/display/FF/Single-Node+Performance+Testing11:23
<awoods>osmandin: thanks11:24
<ajs6f>gregjansen: I was getting a NPE at FedoraTransform line 102: final Session session = sessions.getInternalSession();11:25
which was your add. Any thoughts?
It looks like the way you're getting a session to do some initial setup of types and things for the transform gear.
<gregjansen>ajs6f: looking, yeah
<ajs6f>You sound like BEN!11:26
<escowles>barmintor is back!
<ajs6f>"PATCHES WELCOME!"11:27
barmintor So we're taking charge of some of our own event creation?11:29
<gregjansen>ajs6f: NPE on 102 sounds like a null sessions object, which should have been injected but somehow wasn't.. is this coming up in ITs b/c I haven't seen it myself..11:32
<ajs6f>That's not an injected Session though… It is coming up in ITs, ITs that I have written to create transforms...
But I may be using he API wrong.
So if there's nothing obvious, I'll go away and bang on it some more.
cbeer: Is someone from Stanford involved in the JCR spec process?11:45
<cbeer>ajs6f: not that I know of11:46
<ajs6f>np.
awoods thought there might be.
<gregjansen>ajs6f: If the sessions object (SessionFactory) is null, then it seems like a spring context issue. perhaps the IT doesn't use the master context? the sessionfactory bean is configured in rest.xml
<ajs6f>gregjansen: I'll check that.
gregjansen: I'm not going to do anything until ermadmix can merge what has already gone on.11:47
<gregjansen>ajs6f: let me know when I can help again..11:48
* fcrepo-bot leaves11:49
<cbeer>ajs6f: i thought that's what one of the outstanding pull requests fixed?11:50
ajs6f: where -transform wasn't being scanned in -webapp, etc, etc?
* github-ff joins11:51
[fcrepo4] barmintor closed pull request #193: Version mgr reuse (master...versionMgrReuse) http://git.io/awWTdw
* github-ff leaves
* github-ff joins
[fcrepo4] barmintor created versionMgrReuse (+2 new commits): http://git.io/JBsslA
fcrepo4/versionMgrReuse 6e777dc Benjamin Armintor: allow VersionManager to reuse node to be versioned
fcrepo4/versionMgrReuse 03fd28f Benjamin Armintor: share sessions for change sets; publish FedoraEvents with info sufficient to gen JMS messages
* github-ff leaves
<barmintor>ajs6f: it's in
* github-ff joins
[fcrepo4] barmintor opened pull request #197: Version mgr reuse (master...versionMgrReuse) http://git.io/c2ts5Q
* github-ff leaves
<cbeer>ajs6f: re: JCR, no. maybe awoods was thinking about LDP (where there's no Stanford people involved.. but we could be)11:53
<ajs6f>cbeer: Very possibly, in other contexts. But in the itests in the indexer, I'm not relying on the webapp, I'm building up contexts from the individual modules (since it's itests and not sys tests).
cbeer: Maybe he was. Maybe… he was.11:54
barmintor: on it.
<cbeer>ajs6f: it looks like i might have halved our ingest time by turning on infinispan passivation (store stuff in RAM, don't worry about getting it to disk immediately)... i didn't realize how expensive the ISPN loaders were11:56
<ajs6f>cbeer: holy *&*)&*&)*!
cbeer: Does this make us feel any less good about making claims about "durability"?11:57
cbeer: Like, how long can things stay in heap before getting written out?
or is that adjustable?
<cbeer>ajs6f: i don't know that we have any control
<ajs6f>cbeer: We just decide whether it is instant or not-instant?11:58
<cbeer>ajs6f: but if you really trust your cluster... or you're doing batch ingests and are trying to cook your numbers
<ajs6f>Which we would never do.
<cbeer>oh, definitely not.
<ajs6f>It goes back to the same argument we had with Thorny and Neil in the spring. Absolute reliability and absolute speed are things between which you may have to _choose_11:59
<cbeer>and we certainly wouldn't present it at a conference like CNI
<ajs6f>You can't just decide to have as much of both as you want.
<pivotal-bot>Benjamin Armintor added comment: "This might be done in PR 197" https://www.pivotaltracker.com/story/show/6161756812:03
A. "Horbulaco" Soroka added comment: "This was done several PRs ago. I just forgot to start and finish it." https://www.pivotaltracker.com/story/show/6161756812:06
* travis-ci joins
[travis-ci] futures/fcrepo4#1369 (versionMgrReuse - 03fd28f : Benjamin Armintor): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/6e777dc3d076^...03fd28f03c69
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/15178431
* travis-ci leaves
<pivotal-bot>A. "Horbulaco" Soroka added comment: "Actually, it wasn't quite done. We introduced the new message format to solve this problem in a di..." https://www.pivotaltracker.com/story/show/6161756812:07
A. "Horbulaco" Soroka added comment: "Is there an integration test that specifically shows proper behavior with remove events?" https://www.pivotaltracker.com/story/show/6161756812:08
Benjamin Armintor added comment: "This should also be addressed in PR 197" https://www.pivotaltracker.com/story/show/61482610
Benjamin Armintor finished "Fix o.f.k.o.SimpleObserver so it doesn't blow the heap out with large volumes of objects" https://www.pivotaltracker.com/story/show/61482610
A. "Horbulaco" Soroka added comment: "https://github.com/futures/fcrepo4/pull/197" https://www.pivotaltracker.com/story/show/6148261012:09
Benjamin Armintor finished "Fix JMS reporting for item remove events" https://www.pivotaltracker.com/story/show/61617568
<ajs6f>barmintor: The two commits in that PR seem to me to be entirely independent. I'm not going to squash them— does that make sense with your understanding?12:10
<barmintor>ajs6f: yes
I mean, they have a common goal, but they are 2 different sets of work12:11
awoods: hah https://www.pivotaltracker.com/story/show/5453060612:12
<pivotal-bot>feature: Leverage Event.getMixinNodeTypes() in DefaultFilter (unscheduled) / owner:
A. "Horbulaco" Soroka added comment: "I don't see AtomJMSIT testing this. Can you add a commit to verify that this ticket is resolved in..." https://www.pivotaltracker.com/story/show/6161756812:13
<ajs6f>The Icebox… where good ideas go to get moldy and mildewed.12:14
<pivotal-bot>Benjamin Armintor added comment: "Does HeadersJMSIT not do that?" https://www.pivotaltracker.com/story/show/61617568
A. "Horbulaco" Soroka added comment: "Yes, it does, but this ticket is specifically for the Atom stuff. Or we can just declare the Atom ..." https://www.pivotaltracker.com/story/show/61617568
<bljenkins>Project fcrepo-fixity-corrupter build #528: SUCCESS in 1 min 44 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/528/12:18
<pivotal-bot>Benjamin Armintor added comment: "Right, right, ok." https://www.pivotaltracker.com/story/show/6161756812:19
<gregjansen>cbeer++ the cook is in the kitchen. I trust my cluster.12:21
<ajs6f>cbeer: Would that essentially mean that fcrepo4… (hm, how to say it) _can be shown_ to be faster than fcrepo3, hands-down.12:22
?
<cbeer>ajs6f: that's my hope.
<ajs6f>That would be pleasant.
<cbeer>which is all we want to say, right? in some configurations, you can make fcrepo4 lightning fast.
and in others, you can make fcrepo4 more durable than fcrepo3.
and if you're smarter than us, you might even be able to do both.
<escowles>fcrepo4: fast, durable -- pick at least one12:23
<ajs6f>But no more than three.
<cbeer>escowles: oh, i can do slow and non-durable too.12:24
<escowles>cbeer: but we probably don't want to advertise that as much...
<cbeer>escowles: i think it addresses the flexibility promise.
* ajs6f leaves12:34
* ajs6f joins12:39
* ajs6f leaves12:48
<bljenkins>Project fcrepo-fixity-corrupter build #529: SUCCESS in 1 min 58 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/529/12:56
Project fcrepo-kitchen-sink build #699: SUCCESS in 5 min 54 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/699/13:02
* ajs6f joins13:03
<osmandin>afk 3013:08
<ajs6f>barmintor: Sorry, doing hiring. Do you want me to deal with my weird predilections about type with another commit to the PR? (My Github comment)13:52
<barmintor>type?
Type...
Type...
<ajs6f>The Map generic stuff.
<barmintor>I think the economics of that are bad- the performance penalty is runtime, the compler coaching is compile-time13:53
but if you think it's worth it, add the commit13:54
I'm still working on that test
ajs6f: you were right, btw: no remove ATOM messages13:55
* ermadmix leaves
<barmintor>by which I mean- that other test was testing the wrong pasrt of the code14:00
* ajs6f leaves14:02
* ajs6f joins14:05
barmintor: Do you want to satisfy the remove thing on the legacy message format, or punt? (I have no problem with punting,)14:06
<barmintor>ajs6f: I'm about to push a commit :)
ajs6f: interesting thing I just learned:14:07
when you remove node at /a/b
<ajs6f>barmintor has feat : Lightening Code : Grants +3 to initiative rolls.
<barmintor>you get NODE_REMOVED for /a/b, but you also get PROPERTY_CHANGED /a/lastModifiedBy
<ajs6f>Cheezy.14:08
<barmintor>(I assumed you also get a property message for lastModified)
so… chatty.
<ajs6f>Also cheezy.
Ys.
<barmintor>Chatter cheez.
<ajs6f>I mean, it's true… but just because gossip is true doesn't mean it's not still gossip.14:09
<barmintor>ajs6f: since we can't distinguish Obj/DS nodes, I'm changing the ATOM title to just be "purge"14:10
sound okay?
<ajs6f> Works for me. If you've built a workflow off of the JMS from fcrepo3 that depends on that attribute, and you cavil at its modification… you're going to have bigger problems with fcreop4.14:11
Much bigger problems.14:12
* barmintor is running tests14:16
* barmintor finished tests14:17
* github-ff joins
[fcrepo4] barmintor pushed 1 new commit to versionMgrReuse: http://git.io/OU93LQ
fcrepo4/versionMgrReuse a618ce9 Benjamin Armintor: accomodate NodeRemoved in ATOM/JMS messaging
* github-ff leaves
<ajs6f>on it
barmintor: http://www.theoldfoodie.com/2013/12/weird-beers.html14:26
Beer of Sulphuric Acid.
Nuff said.
barmintor: Getting weird IT errors on the JMS stuff:14:28
https://gist.github.com/ajs6f/7879282
looks like a parsing problem?
<barmintor>What? I didn't get that error.
<ajs6f>(From your latest commit)
Then it must not be real
<barmintor>I demand that you run the test again, so that we can revel in a new, intermittent error.14:29
<ajs6f>Travis doesn't run ittests, and I haven't seen any barking from jenkins.
I ran it thrice.
<barmintor>THRICE?
<ajs6f>And I'll do it again. QUICE!
(That _should_ be a word.(
<osmandin>I thought you guys were getting rid of the message body?14:30
<ajs6f>osmandin: This is backfill.
* jcoyne joins
<osmandin>ajs6f: backfill, meaning?
<ajs6f>osmandin: barmintor was trying to keep the Atom stuff alive.
is trying
<barmintor>ajs6f: I think I got the error, but I had to run that test 10 or 12 times14:32
* travis-ci joins14:33
[travis-ci] futures/fcrepo4#1371 (versionMgrReuse - a618ce9 : Benjamin Armintor): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/03fd28f03c69...a618ce9b4302
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/15185851
* travis-ci leaves
<ajs6f>barmintor: Maybe your JVM has a thicker skin?
barmintor: I tried it for the fourth time and didn't get it.
<barmintor>ajs6f: Unexpected close tag </s>; expected </summary>.
<ajs6f>AWESOME!
Did we ever merge osmandin's library upgrade?
<barmintor>feels like a problem with killing the listener
<ajs6f>Seriously, osmandin went through hell with the Atom parser. Did we ever merge his fix?14:34
'Cause that miht be this.
Kill ing the listener?
* jcoyne leaves
<barmintor>like something is truncating content14:35
<ajs6f>Hm.
But that's not how JMS listeners behave in my experience. they don't cut messages short when they're destroyed. But I may just have never happened to have seen that.
<osmandin>I think the proposed fix was not used since we were going to use just headers/properties (no parsing)14:37
afk14:40
<ajs6f>We should have know.14:41
known.
"The past is never dead. It's not even past." — Faulkner, Requiem for a Nun14:42
<bljenkins>Project fcrepo-fixity-corrupter build #530: SUCCESS in 1 min 43 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/530/14:43
<barmintor>yeah, those errors are fake14:45
I bet something weird is happening because the junit @After method shuts down the JMS connex
but I' get, e.g., " com.ctc.wstx.exc.WstxParsingException: Unexpected close tag </nuri>; expected </name>."14:46
and the text o that message does not actually have the String "</nuri>" in it
<ajs6f>There are more subtle ways to shutdown the JMS connex than what we're doing in that method.14:47
<bljenkins>Project fcrepo-kitchen-sink build #700: SUCCESS in 3 min 49 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/700/
<barmintor>Yes… more subtle, and more nefarious… muahahahaHAHAHAHAHAAHA
<ajs6f>Are we evil? I mean, I'm down with that. I just want to know.15:03
* github-ff joins15:04
[fcrepo4] barmintor pushed 1 new commit to versionMgrReuse: http://git.io/ksGJhA
fcrepo4/versionMgrReuse 24169f2 Benjamin Armintor: stop listening for JMS messages when a test is over
* github-ff leaves
<pivotal-bot>Mike Durbin added "Create "Documentation > Features" tree in wiki." https://www.pivotaltracker.com/story/show/6218140215:10
Mike Durbin started "Create "Documentation > Features" tree in wiki." https://www.pivotaltracker.com/story/show/62181402
<bljenkins>Project fcrepo-fixity-corrupter build #531: SUCCESS in 1 min 21 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/531/15:32
<ajs6f>barmintor: Hate to say it, but on the fourth run… I saw it again:15:34
https://gist.github.com/ajs6f/7880292
parsing exception.
Looks similar in character.
<barmintor>But it's a different error
<ajs6f>Yes. But a parsing problem.
Should we try to integrate osmandin's work?
<barmintor>Yes, but a *different* parsing problem. It's intermittent and vairable!15:35
ajs6f: yes, probably
<ajs6f>Should we just chuck the Atom crap out the window, crank the Molly Hatchet, and drive off into the sunset?
<barmintor>Ask awoods
that is just nonsense in the test machinery, though15:36
<ajs6f>barmintor: k. You want to do the branch-to-branch merge, or you want to try the merge osmandin's branch first, then rebase yours approach.
?
<bljenkins>Project fcrepo-kitchen-sink build #701: SUCCESS in 4 min 12 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/701/
<barmintor>(B)
<ajs6f>k
osmandin: We're going to try to merge your branch that corrects Atom parsing. Stand by for mayhem.
osmandin: Urg. Merge problems. You got time to help?15:37
I'm busy eating bean tacos.15:38
osmandin: can you rebase?
<osmandin>ajs6f: sure, I can, but probably not today... (also, the code would first need to be reviewed by awoods?)15:39
<ajs6f>osmandin: Cool. I thought awoods had reviewed it already?15:40
<osmandin>ajs6f: I'm not sure
<pivotal-bot>Esme Cowles finished "Single-node F4 Ingest Benchmarks" https://www.pivotaltracker.com/story/show/6055694815:41
<ajs6f>osmandin: Okay. I'll review and you can rebase when you have a chance.
awoods: You're okay with that, right?
awoods: Great.
awoods: np.
<pivotal-bot>Esme Cowles added comment: "When using "contentBasedSha1" : "false" option, we can serve files as large as 1TB without problems. See h..." https://www.pivotaltracker.com/story/show/6177364416:03
Esme Cowles finished "Determine filesize read limit via REST API" https://www.pivotaltracker.com/story/show/61773644
* fcrepo-bot joins16:30
<pivotal-bot>Mike Durbin added "Create "Documentation DRAFT > Feature Walkthrough" wiki page." https://www.pivotaltracker.com/story/show/6218834216:37
Mike Durbin edited "Create "Documentation DRAFT > Feature Walkthrough" wiki page." https://www.pivotaltracker.com/story/show/62188342
<mikeAtUVa>ajs6f, cbeer: you both appear to have had a hand in wiki pages that talk about the fedora namespace in CND.16:46
In one page its reported that properties with those namespaces are immutable, and in another that they shouldn't be modified.16:47
cbeer, ajs6f: I broke this convention and wonder if I should fix my code or "fix" the documentation.
<cbeer>mikeAtUVa: with what?
<mikeAtUVa>I introduced a property: fedora:versioningPolicy that the user is meant to update directly.16:48
<ajs6f>is that meant to record a resource-specific choice?
<cbeer>mikeAtUVa: maybe change it to fedora-config:versioningPolicy?
<ajs6f>what cbeer said, or if it's global, move it to a node in the config section of the repo.16:49
<mikeAtUVa>ajs6f: yes, though I have instructions on how to update the CND to auto-populate it for a repository-wide setting.
<ajs6f>Hm. Maybe there are two settings here that interact?
One is global, the other is local. The semantics coincide?
<mikeAtUVa>cbeer: what URI prefix should we use for fedora-config:?16:50
ajs6f: the property is specific to a node, not set at some global scope... I wasn't aware of global property scopes or how to set them and it's likely not in my best interest to change the interaction now.16:51
<ajs6f>The property, sure. I'm trying to understand what the behavior at stake is. It sounds like something that normally has a default, which sometimes one wants to override locally?16:52
<mikeAtUVa>cbeer, ajs6f: obviously it woul be eaiser for me to update the wiki documentation, but if I grossly misued our namespaces I'm happy to fix it before this release.
<ajs6f>I think it's better to be "doctrinaire" about that namespace.
It's easy to introduce another
* ajs6f leaves16:58
* osmandin leaves17:00
<pivotal-bot>Mike Durbin added "Reoganize performance documentation into "Documents DRAFT > Performance"" https://www.pivotaltracker.com/story/show/6219136417:05
Mike Durbin edited "Reoganize performance documentation into "Documents DRAFT > Performance"" https://www.pivotaltracker.com/story/show/6219136417:06
Mike Durbin added "Create "Documentation DRAFT > Glossary" page" https://www.pivotaltracker.com/story/show/6219153617:08
Mike Durbin edited "Create "Documentation DRAFT > Glossary" page" https://www.pivotaltracker.com/story/show/62191536
Mike Durbin added "Use new namespace for fedora:versioningPolicy" https://www.pivotaltracker.com/story/show/6219195817:12
* ermadmix joins17:20
* github-ff joins17:28
[fcrepo4] mikedurbin opened pull request #198: Updated versioningPolicy property to use a new namespace. (master...versioning-policy-namespace) http://git.io/T6accw
* github-ff leaves
<pivotal-bot>Mike Durbin started "Use new namespace for fedora:versioningPolicy" https://www.pivotaltracker.com/story/show/62191958
Mike Durbin added comment: "https://github.com/futures/fcrepo4/pull/198" https://www.pivotaltracker.com/story/show/6219195817:29
Mike Durbin finished "Create "Documentation > Features" tree in wiki." https://www.pivotaltracker.com/story/show/62181402
Mike Durbin finished "Use new namespace for fedora:versioningPolicy" https://www.pivotaltracker.com/story/show/62191958
Mike Durbin started "Create "Documentation > Features" tree in wiki." https://www.pivotaltracker.com/story/show/62181402
* mikeAtUVa leaves
* fcrepo-bot leaves17:39
* ksclarke leaves18:06
* ermadmix leaves19:40
* barmintor_ joins19:53
* barmintor leaves19:57
<cbeer>it's probably fair to make fcrepo3 calculate datastream checksums in order to do a head-to-head comparison with fcrepo4, right?20:29
* ermadmix joins20:39
* ksclarke joins20:55
* ermadmix leaves21:10
* ermadmix joins21:26
* ermadmix leaves21:59
* ermadmix joins22:31
* ermadmix leaves23:36
* ermadmix joins23:43
* ermadmix leaves00:15
* ermadmix joins00:55
* awoods leaves
* ksclarke leaves01:15
* ermadmix leaves01:27
* ermadmix joins01:29
* ermadmix leaves02:01
* kaarefc joins02:11
* ermadmix joins02:37
* ermadmix leaves03:09
* ermadmix joins03:36
* ermadmix leaves04:08
* ermadmix joins04:19
* ermadmix leaves04:51
* ermadmix joins04:54
* escowles leaves05:08
* escowles joins05:09
* escowles leaves05:18
* escowles joins05:19
* ermadmix leaves05:26
* ermadmix joins05:45
* ermadmix leaves06:17
* ermadmix joins06:58
* ermadmix leaves07:30
* ermadmix joins07:50
* ermadmix leaves08:22
* ermadmix joins08:25
* ermadmix leaves08:57
* ksclarke joins09:00
* ermadmix joins09:15
* osmandin joins09:21
* mikeAtUVa joins09:26
* awoods joins09:32
* awoods leaves
* awoods joins09:33
* tecoripa joins
* tecoripa leaves09:34
<pivotal-bot>Esme Cowles started "A page to the Wiki describing recommend solutions to ingesting files of varying sizes." https://www.pivotaltracker.com/story/show/5506912609:40
* ajs6f joins09:43
<pivotal-bot>Esme Cowles added "Inventory Beta Design Documents for Documentation Draft" https://www.pivotaltracker.com/story/show/6223474609:45
Esme Cowles started "Inventory Beta Design Documents for Documentation Draft" https://www.pivotaltracker.com/story/show/62234746
Esme Cowles edited "Inventory Beta Design Documents for Documentation Draft" https://www.pivotaltracker.com/story/show/62234746
Andrew Woods delivered "fcr:transform exception" https://www.pivotaltracker.com/story/show/6194263809:50
* github-ff joins
[fcrepo4] awoods closed pull request #191: addded org.fcrepo.transform to rest.xml component scan (master...erj-transform) http://git.io/qtFHcQ
* github-ff leaves
<pivotal-bot>Andrew Woods edited "move fcrepo-auth-roles-common into fcrepo" https://www.pivotaltracker.com/story/show/6201958609:51
Andrew Woods edited "Fix JMS reporting for item remove events" https://www.pivotaltracker.com/story/show/6161756809:53
* travis-ci joins10:08
[travis-ci] futures/fcrepo4#1377 (master - f46f103 : Andrew Woods): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/d64e7d3cd085...f46f10310463
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/15229232
* travis-ci leaves
<awoods>escowles: I have some inline comments on: https://wiki.duraspace.org/display/FF/Single-Node+Performance+Testing10:13
<pivotal-bot>Andrew Woods delivered "Use new namespace for fedora:versioningPolicy" https://www.pivotaltracker.com/story/show/6219195810:15
* github-ff joins
[fcrepo4] awoods closed pull request #198: Updated versioningPolicy property to use a new namespace. (master...versioning-policy-namespace) http://git.io/T6accw
* github-ff leaves
<escowles>awoods: do you want answers inline? or maybe we should switch to comments at the bottom of the page, or email?10:16
<awoods>escowles: or IRC? I am around for the next couple of hours10:17
<bljenkins>Project fcrepo-fixity-corrupter build #532: SUCCESS in 2 min 23 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/532/10:19
<escowles>awoods: for the first comment (re: ingest slowing down, presumably because of nodes were all under a single parent): yes, this is probably the case -- this combined with the filesystem federation showing the same issue means we should highlight this
esp. since having all objects under a single node is the fcrepo3 pattern, we should make sure that people understand they will hit performance problems with that pattern in fcrepo410:20
re: why the combined results matrix: my intention was to have a snapshot of all the benchtool results over time as the code changed -- a holistic view so you could see, for example, if a change improved ingest perf but at the expense of hurting reads, etc.10:21
<awoods>escowles: I agree. Maybe that line in the wiki can state something to that effect, instead of just saying that "performance was degraded"10:22
escowles: the evolving test results makes sense...10:23
<escowles>yes, that makes sense -- and i think in general all of the design/testing pages need to have a read-through with an eye to adding context and practical advice like that
* ajs6f leaves
<escowles>for the #A v. #C: the page linked to has several tests, numbered #A to #F -- the numbers i pulled out for the chart and #A and #C -- maybe saying that is unnecessary, or there's a clearer way to say it?10:24
<awoods>escowles: I was just wondering why one table has read/update/delete combined, and the other tests have the same information in separate tables.
<escowles>awoods: again, a sentence explaining why would help
<awoods>escowles: the reason is still not clear to me10:25
* ajs6f joins
ermadmix: Are you good to go as far as questions about that indexer PR?
<awoods>escowles: The final table is called "combined"... why are the tables before it not "combined"?10:26
<escowles>awoods: we could combine most of the previous tables into the combined table -- the only thing that wouldn't make sense would be the ingest table (we have a lot of ingest-only results)10:27
<awoods>escowles: That could work...10:28
escowles: Is this the page that will eventually contain the summarized results? or will that be a new parent page?10:29
* ajs6f leaves
<escowles>awoods: i think there will be a new parent (and in general i think this is the case for most of the design pages too)10:30
<awoods>escowles: Great. Then that parent page will reference the details in this current page.
* travis-ci joins
[travis-ci] futures/fcrepo4#1378 (master - efce262 : Andrew Woods): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/f46f10310463...efce262e5cdf
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/15230573
* travis-ci leaves
<escowles>so the top-level page can be a single table with summarized results, where people can add their own numbers, and then link them to a wiki page for each set of testing (where all my testing in the last few sprints would be a single page, for example)10:31
<awoods>escowles: Perfect. Do you have thoughts on where that parent page will live?
* ajs6f joins
<escowles>awoods: i think it should be under the admin guide
<awoods>escowles: Sounds good. Do you also plan on "graduating" the existing "Test - *" pages up to the admin guide as well?10:32
<escowles>i was thinking of a section that would be a summary of all the performance-related issues (large files, only so many objects under a single node, clustering, how config options affect perf, etc.)
and the test - * pages would live under that
<mikeAtUVa>escowles, awoods: I summarized awoods' gorganizational goals for wiki performance stuff in a pivotal ticket: https://www.pivotaltracker.com/story/show/6219136410:35
<pivotal-bot>feature: Reoganize performance documentation into "Documents DRAFT > Performance" (unstarted) / owner:
<ajs6f>cbeer: Did the IRC logger go offline? There seems to be no page for today...
<pivotal-bot>Mike Durbin edited "Reorganize performance documentation into "Documents DRAFT > Performance"" https://www.pivotaltracker.com/story/show/62191364
<awoods>escowles: sounds like a plan.
<pivotal-bot>Andrew Woods edited "Reorganize performance documentation into "Documents DRAFT > Performance"" https://www.pivotaltracker.com/story/show/6219136410:36
<escowles>afk
<awoods>mikeAtUVa: re: 62191364, nice
<pivotal-bot>Andrew Woods added comment: "Will "Deliver" when inline comments have been removed/addressed." https://www.pivotaltracker.com/story/show/6055694810:39
Andrew Woods rejected "Single-node F4 Ingest Benchmarks" https://www.pivotaltracker.com/story/show/6055694810:40
* tecoripa joins
* ajs6f leaves10:45
* ajs6f joins10:49
<ermadmix>A thought re escowles [10:20], would it make sense for the fcrepo3 pattern in fcrepo4 (having a all under objects node), rather adopt the way objects are stored in the fedora 3 backend (an akubra like hashing out of directory paths) by having an endpoint alias (a pid) reference another endpoint (a hashed out path)
<bljenkins>Project fcrepo-fixity-corrupter build #533: SUCCESS in 3 min 31 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/533/10:50