Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* fcrepo-bot leaves00:01
* ksclarke leaves00:29
* ksclarke joins00:45
* dwilcox joins01:13
* dwilcox leaves01:47
* ksclarke leaves01:51
* dwilcox joins02:25
* dwilcox leaves02:36
* f4jenkins leaves04:09
* f4jenkins joins04:10
* fasseg joins04:33
* fasseg leaves07:01
* awead joins08:28
* mohamed joins08:38
* mohamed leaves08:48
* mohamed joins08:51
* mikeAtUVa joins09:26
* gregjansen joins09:30
* mohamed1 joins09:31
* mohamed leaves
* ajs6f joins09:41
cbeer: If we aren't doing the paging on top of the iterators _now_, how are we doing it?09:42
* ajs6f leaves09:47
* ajs6f joins09:48
awoods: ping09:54
<ajs6f>re: https://www.pivotaltracker.com/story/show/59603190
<pivotal-bot____>chore: Use Maven parallelism in CI (unscheduled) / owner: A. "Banjaxel" Soroka
<ajs6f>Can we set up an auxiliary build in CI with -T 2 just to keep an eye on what fails?09:55
We should be able to parameterize the current build, then have two forms, one with -T 1 and one with -T 2.09:56
<awoods>ajs6f: sure... may so as to not over-tax that machine on each commit, we could run it once a night.
<ajs6f>(I'm talking about Jenkins, not Travis.)
awoods; Sure. That would be fine.
<awoods>ajs6f: does -T 2 work for you presently?09:57
<ajs6f>awoods: it would be interesting to see what fails with two builds running at the same time. Every failure would represent some possible strengthening in the test framework.
* ajs6f1 joins09:58
* ajs6f leaves
<ajs6f1>awoods: Just tried tests, now trying itests.
Tests were fine.
<awoods>ajs6fn: good09:59
<ajs6f1>awoods: itests were good, too (I'm honestly surprised) but there were a huge number of random stacktraces on shutdown for fcrepo-webapp. Problems with the container management for its itests, I think. I'll be curious to see what it looks like on CI.10:03
* awead leaves
<ajs6f1>but the itests completed successfully10:04
I think I'll try -T 4, just for the hell of it.
<awoods>ajs6f1: that is surprisingly good news. I will be interested to hear the results of -T 4. As noted in the ticket, using -T 2 on my box was not as successful.10:05
<ajs6f1>tests worked, and screamingly fast (tho' I could almost see my battery declining).
awoods: Yes, but that was months ago, and fcrepo-rss was the culprit.10:06
<awoods>ajs6f1: I will try again in a moment... you are Mac, right?
<ajs6f1>awoods: Yep.
awoods: The thing I like about Maven multithreading is that it allows task like JavaDoc generation, license checking, etc. to go on in parallel. And that's where you get a real speed boost.10:07
Some of that stuff really takes a surprising amount of time.
awoods: with -T 4, 1 http-api itest failed:10:09
Failed tests:10:10
FedoraNodesIT.testBreakFederation:1811->AbstractResourceIT.createObject:237 expected:<201> but was:<409>
awoods: Not bad, really.
awoods: I think it's worth trying to fix it. I'm guessing it's a matter of tinkering with the Spring test framework.10:11
<awoods>ajs6f1: thanks for pushing on this. Have you noticed the intermittent failures on Jenkins as well?
<ajs6f1>awoods: if we can get to a place where we normally assume that multithreaded builds work, we have an additional layer of safety.
awoods: No, I haven't been watching Jenkins. Are they similar?10:12
<f4jenkins>Project fcrepo-message-consumer build #658: ABORTED in 12 hr: http://jenkins.fcrepo.org/job/fcrepo-message-consumer/658/
<ajs6f1>Well, there's that. ^^^
But that's not a repo build.
<awoods>ajs6f1: no, fcrepo-message-consumer sometimes hits OOM10:13
<ajs6f1>awoods: Really? OOM?
That's kind of weird.
<awoods>ajs6f1: the java/maven settings just need to be dialed on Jenkins for that project10:14
<ajs6f1>awood: oh, okay. Although that build really shouldn't take much heap.
operation of the consumer shouldn't take much heap. We'd like it to be a very light process.
<awoods>ajs6f1: It is the tests, it starts: fuseki, jetty, along with f4 in addition to the message-consumer webapp10:15
<ajs6f1>awood: oh, right. It's not just operating the consumer. It's operating a repo, too.
* ksclarke joins
<awoods>ajs6f1: fyi10:16
ajs6f1: the other projects are pretty solid10:17
<ajs6f1>awoods: The other projects meaning everything but those two?10:19
<ajs6f1>I wonder what the story with auth-rbacl is. Maybe it's just really hard to figure out how to pronounce it.10:20
Who can say?
<awoods>ajs6f1: pronunciation is likely the issue10:22
<ajs6f1>awoods: There must be a Jenkins plugin.
* awead joins10:23
* awoods thinking ajs6f1 is a true comedian
<ajs6f1>awoods: No, just the kind of person who cracks jokes to hide nervousness.
Fedora makes me nervous.
<awoods>ajs6f1: keep the comedy coming... and maybe consider meditation10:24
<ajs6f1>awoods: I've been working on the following koan: what is the fixity of an empty bitstream…?10:25
awoods: I think the answer is: no one does proper fixity checking anyway.10:26
* ajs6f joins10:29
* ajs6f1 leaves
<ajs6f>awoods: Ran mvn -T 4 verify again, and got: Failed tests:
FedoraNodesIT.testBreakFederation:1811->AbstractResourceIT.createObject:237 expected:<201> but was:<409>
That makes me happy, because it seems to point at the weakness for multithreading being in the test framework, and not in individual tests.
409 is CONFLICT, so it sounds like we've got itests trying to use the same identifiers and crashing into each other when they run in paralell.10:30
I've been meaning to go through our itests and make them all use random UUIDs for identifiers.10:31
* dwilcox joins10:32
<awoods>ajs6f: we are pretty good on the use of UUIDs in the ITs, but apparently not all the way home.
<ajs6f>awoods: Right. _If_ my hypothesis is correct.
Anyway, we should make sure.10:33
<pivotal-bot____>A. "Banjaxel" Soroka added "Use ids for i-tests that cannot collide" https://www.pivotaltracker.com/story/show/7900920810:34
* ajs6f leaves
* scossu joins10:44
* whikloj joins10:48
<pivotal-bot____>Jared Whiklo added comment: "https://github.com/fcrepo4/fcrepo4/pull/473 - Again" https://www.pivotaltracker.com/story/show/6150042610:50
Jared Whiklo finished "If someone tries to create a new node with an unregistered namespace in the HTML UI, capture that error and give th�" https://www.pivotaltracker.com/story/show/61500426
* ajs6f joins
* awead leaves10:59
* ajs6f leaves11:00
* ajs6f joins11:02
<cbeer>ajs6f, awoods ; since you mentioned auth-rbacl earlier, i have a gripe about it from my refactoring..11:03
it's a dependency of fcrepo-webapp
* awead joins11:04
<cbeer>but because it's in a separate project, you sometimes have to build twice to get fcrepo4 to work right... build fcrepo4, build rbacl, and then rebuild fcrepo4.
<ajs6f>Should _any_ authz stuff be in webapp by default?
<cbeer>ajs6f: beats me. it doesn't seem like a core concern (and clearly someone thought enough to move it to a different git repository), but it's still baked in.
<ajs6f>cbeer: I think that's the prob. Break the dependency.
<cbeer>ajs6f: stack level too deep. today i need to try to get the LDP HTTP API actually working.11:06
<ajs6f>cbeer: +1. I just mean that _someone_ should do it.
<cbeer>awoods: is there a TWG call today?11:14
i see an agenda, but i don't remember an email11:15
<awoods>cbeer: yes11:18
cbeer/ajs6f: mikeAtUVa is working the removal of the auth dependencies from fcrepo-webapp: https://www.pivotaltracker.com/story/show/7434639611:19
<pivotal-bot____>feature: Create authz-configurable webapp (unstarted) / owner: Mike Durbin
<ajs6f>mikeAtUVa is sitting ten feet from me suffering through the same meeting I am.11:20
<ajs6f>Looking very, very, titred.
<awoods>apparently mikeAtUVa is "under the weather"
<ajs6f><wry>Something like that.</wry>
* ajs6f leaves11:56
* ajs6f joins11:58
* barmintor joins12:00
barmintor cbeer ^^
barmintor, ajs6f, escowles: after the call, do you have a little time to stay on the call and listen to me ramble about LDP?12:08
<barmintor>cbeer: a brief ramble
<ajs6f>cbeer: Sorry, no, but maybe we can catch up later today? Are you around at 4 Eastern?
<cbeer>ajs6f: got another call then.12:09
barmintor: brief is fine. as long as your gut reaction isn't that i'm totally crazy, i'm happy to press forward.
<escowles>cbeer: i was planning on talking with awoods after the call...
<ajs6f>cbeer: Well, you'll get more use out of barmintor than me.
* github-ff joins12:21
[fcrepo4] whikloj opened pull request #478: RDF'ed serialization (master...fcrepo-rdf-serialize) http://git.io/uL7JAQ
* github-ff leaves
<pivotal-bot____>Jared Whiklo added comment: "https://github.com/fcrepo4/fcrepo4/pull/478" https://www.pivotaltracker.com/story/show/65221404
Jared Whiklo finished "RDF-ify serialization formats" https://www.pivotaltracker.com/story/show/65221404
Jared Whiklo started "Acceptance test: Insert/delete object in hierarchy" https://www.pivotaltracker.com/story/show/7882311412:30
* dwilcox leaves12:34
<pivotal-bot____>A. "Banjaxel" Soroka started "Use ids for i-tests that cannot collide" https://www.pivotaltracker.com/story/show/7900920812:44
* github-ff joins12:45
[fcrepo4] ajs6f opened pull request #479: Improving the use of identifiers for integration tests (master...IdsForItests) http://git.io/CrXo8A
* github-ff leaves
<ajs6f>awoods: That ^^^ includes just one example, to show a way to to generate good pids via reflection. Tell me what you think.
* ksclarke leaves12:59
* mikeAtUVa leaves
* gregjansen leaves
* mickeroo leaves
* ruebot leaves
* barmintor leaves
* whikloj leaves
* f4jenkins leaves13:00
* ajwagner leaves
* pivotal-bot____ leaves
* pmurray_away leaves
* awoods leaves
* cbeer leaves
* escowles leaves
* bljenkins leaves
* Jotudin leaves
* barmintor joins13:01
* whikloj joins
* ksclarke joins
* gregjansen joins
* mikeAtUVa joins
* f4jenkins joins
* ruebot joins
* mickeroo joins
* ajwagner joins
* pivotal-bot____ joins
* awoods joins
* cbeer joins
* pmurray_away joins
* escowles joins
* bljenkins joins
* Jotudin joins
* ksclarke leaves
* mikeAtUVa leaves
* ksclarke joins
* mikeAtUVa joins
* gregjansen leaves
* mickeroo leaves
* gregjansen joins13:02
* mickeroo joins
* ajs6f leaves13:04
* gregjansen leaves13:38
<cbeer>awoods: can you be sure to look at https://github.com/fcrepo4/fcrepo4/pull/475 today? it's going to be another one that creates nasty merge conflicts if it isn't merged earlier13:40
barmintor: thanks. i feel like i've had so much of the kool-aid it's hard to tell what's still reasonable.13:43
<barmintor>cbeer: I know this feeling.
but I also cannot overstate the value of being able to say: “Fedora doesn’t really have a special API”
* awead leaves13:45
* awead joins13:47
* fcrepo-bot joins
* github-ff joins13:49
[fcrepo-module-auth-rbacl] mikedurbin opened pull request #3: Updated spring configurations to reflect fcrepo4 changes. (master...shutdown-hook) http://git.io/l3GD6Q
* github-ff leaves
* github-ff joins13:53
[fcrepo-message-consumer] mikedurbin opened pull request #53: Updated spring configurations to reflect fcrepo4 changes. (master...shutdown-hooks) http://git.io/V_qsNg
* github-ff leaves
* github-ff joins13:56
[fcrepo-module-auth-xacml] mikedurbin opened pull request #24: Updated spring configurations to reflect fcrepo4 changes. (master...shutdown-hook) http://git.io/3k5g7Q
* github-ff leaves
<pivotal-bot____>Mike Durbin added comment: "https://github.com/fcrepo4/fcrepo-message-consumer/pull/53
https://github.com/fcrepo4/fcrepo-module-auth-rbacl�" https://www.pivotaltracker.com/story/show/66454096
<awoods>cbeer: re: PR-475, I will look at it today.14:31
* scossu leaves14:43
* fcrepo-bot leaves14:52
* scossu joins15:05
<pivotal-bot____>Jared Whiklo added comment: "Assuming I did not miss something with this test, here is the results.15:26
https://wiki.duraspace.org/display/FF/�" https://www.pivotaltracker.com/story/show/78823114
Jared Whiklo finished "Acceptance test: Insert/delete object in hierarchy" https://www.pivotaltracker.com/story/show/78823114
* ajs6f joins15:34
* awead leaves15:39
* gregjansen joins15:53
* awead joins15:59
* mikeAtUVa leaves16:15
* ajs6f1 joins16:22
* ajs6f leaves
<ajs6f1>awoods: ping16:28
<ajs6f1>awoods: have you noticed that our javadocs generation is being run between test and i-test (test and verify, in Maven parlance)? Shouldn't that be happening only for the "site" phase?
Isn't that slowing down builds needlessly?
<barmintor>not if the unit tests modify the codebase16:31
<awoods>ajs6f1: agreed, we never altered the default setup, apparently
barmintor: really?
<barmintor>awoods: the unit tests do not modify the codebase, but I couldn’t be so agreeable to ajs6f all day
<awoods>barmintor: I was about to say...16:32
* ajs6f1 leaves
* ajs6f joins
<awoods>barmintor: you blew ajs6f1 out of the IRC
<ajs6f>barmintor: Why would that require rebuilding javadocs locally? Certainly it means they changed, but does it follow that you want to rebuild them? Checkstyle does its audit on the code itself, right?
ajs6f: I was teasing you
<ajs6f>We're better off with that punk ajs6f1 gone.16:33
barmintor: {sigh} SHould have known.
<awoods>ajs6f: is ajs6f1 your evil twin?
<awoods>ajs6f: be that as it may, please create a ticket to push javadocs to site16:34
ajs6f: or better yet, just push your PR
<pivotal-bot____>A. "Banjaxel" Soroka added "Move JavaDocs generation to Maven "site" phase" https://www.pivotaltracker.com/story/show/79040178
<awoods>cbeer: with PR-475, I am getting an IT failure in fcrepo-http-api16:37
FedoraBatchIT.testModifyMultipleDatastreams:105 Didn't find the object title!
<ajs6f>out for the day
* ajs6f leaves
<pivotal-bot____>A. "Banjaxel" Soroka edited "Move JavaDocs generation to Maven "site" phase" https://www.pivotaltracker.com/story/show/7904017816:38
Andrew Woods added comment: "Pending code review comment responses." https://www.pivotaltracker.com/story/show/6645409617:08
Andrew Woods rejected "Ensure a clean shutdown of the repository on container shutdown." https://www.pivotaltracker.com/story/show/6645409617:09
* mohamed1 leaves17:20
<pivotal-bot____>Andrew Woods added comment: "Some initial comments in code review, but unable to complete review as there were conflicts when I performed �" https://www.pivotaltracker.com/story/show/6150042617:25
Andrew Woods rejected "If someone tries to create a new node with an unregistered namespace in the HTML UI, capture that error and give th�" https://www.pivotaltracker.com/story/show/61500426
* awead leaves17:26
* dwilcox joins17:32
<whikloj>cbeer: Can you give me some background on this: https://www.pivotaltracker.com/story/show/6088547217:37
<pivotal-bot____>feature: Support pre-conditions on fcr:datastreams (unstarted) / owner: Jared Whiklo
<whikloj>cbeer: Not sure exactly what you want, also seeing some sort of "doesnt-match" header on deletes
<pivotal-bot____>Andrew Woods added comment: "Pending response to code review comments." https://www.pivotaltracker.com/story/show/6522140417:38
Andrew Woods rejected "RDF-ify serialization formats" https://www.pivotaltracker.com/story/show/65221404
Andrew Woods delivered "Acceptance test: Insert/delete object in hierarchy" https://www.pivotaltracker.com/story/show/7882311417:46
Andrew Woods added comment: "@cbeer: With your present LDP refactoring, should we hold off on this ticket?" https://www.pivotaltracker.com/story/show/6088547217:47
<cbeer>awoods: yes, we probably should.
<awoods>whikloj ^^17:48
<whikloj>awoods: So noted, moving on.
<cbeer>awoods: in general, anything that says FedoraNodes, FedoraContent, Datastream, Resource or Object is probably a bad idea to touch.17:49
<awoods>cbeer: ok. Also, did you see that my review of PR-475 was stopped by an IT failure?17:50
<cbeer>awoods: i saw that. i'm building now and don't seem to have had a problem.
awoods: yep, no problems.17:55
<awoods>cbeer: let me run it again... for the third time.
<cbeer>i'm pulling up the test now and will see if anything jumps out at me
looking at it, i think it /should/ be broken.17:57
<awoods>cbeer: does that mean that the test is testing a failure case, or that you are surprised it passes when you run the test?17:58
<cbeer>wait, no, it should work.
https://github.com/fcrepo4/fcrepo4/pull/475/files#diff-3cd9c4d0ac7b48b76f2689212b21bc44R311 should be providing the object title.17:59
<awoods>cbeer: same failure, same spot18:03
FedoraBatchIT.testModifyMultipleDatastreams:105 Didn't find the object title!
<cbeer>i've kicked off a build on a linux box
<awoods>cbeer: I will be interested to hear the results18:04
* whikloj leaves
<cbeer>awoods: yep, i get the same failure there... sun java vs openjdk, and slightly different 1.7 versions.18:08
* ksclarke leaves18:09
<awoods>cbeer: that is not nice
cbeer: I need to step away for ~2hours in just a moment
<cbeer>awoods: ok. i'm going to try to poke at it, but i might need you to pull out the debugger.18:10
<awoods>cbeer: keep me posted. Are you adverse to running the debugger?
<cbeer>awoods: figuring out how to remotely debug sounds more painful than waiting for you to do it :)18:11
<awoods>cbeer: I am happy to do it... but at some point it is a trick you may like to have up your sleeve.18:12
<cbeer>ok, weird. i can recreate it in intellij18:14
<awoods>bbi ~2hours18:17
<cbeer>might need ajs6f to chime in, it seems like something odd is happening in the RdfStream?18:20
and now it doesn't fail.18:22
* awead joins18:23
* awead leaves18:28
<cbeer>yeah, i can't get it to break again... but there's so much iterator streaming going on...18:29
and works in my linux environment too18:30
awoods: i wonder if there's something stale in one of our environments.
guess i'll try throwing away my .m2
works beautifully everywhere.18:48
* dwilcox leaves18:57
* scossu leaves19:14
* whikloj joins19:20
* whikloj leaves19:27
<awoods>cbeer: I was so hopeful... but after nuking my .m2/repository, I am still getting the same failure.21:25
<cbeer>and now i can't recreate it anywhere.21:27
<awoods>cbeer: let me see if Jenkins can run it.21:28
<f4jenkins>Project fcrepo4-experiment build #1: FAILURE in 8 min 3 sec: http://jenkins.fcrepo.org/job/fcrepo4-experiment/1/21:40
<awoods>cbeer: that is actually promising21:42
* f4jenkins leaves21:57
* ksclarke joins
* f4jenkins joins
* f4jenkins leaves21:58
* f4jenkins joins
Yippee, build fixed!22:07
Project fcrepo4-experiment build #4: FIXED in 8 min 23 sec: http://jenkins.fcrepo.org/job/fcrepo4-experiment/4/
* whikloj joins22:13
* awead joins
* awead leaves22:14
<whikloj>awoods: ping22:19
<whikloj>awoods: Do you and ajs6f want the IS_IN_FORMAT property in the RdfLexicon changed to the dc:format, or a new one added?22:20
<awoods>whikloj: IS_IN_FORMAT only exists in your PR, no?
whikloj: and dc:format exists in the wild22:21
whikloj: in which case, you should be able to remove IS_IN_FORMAT from your PR and just use the standard dc:format without adding anything to the RdfLexicon.22:22
<whikloj>Giulah added IS_IN_FORMAT to the RdfLexicon in her code.
<awoods>whikloj... which is now your code.
whikloj: it can be dropped in favor of dc:format
<whikloj>awoods: right so I can dump it and just reference it
awoods: ok thanks22:23
<awoods>whikloj: no go to sleep
<whikloj>awoods: ha, I only have till midnight before my time is up on this
<awoods>whikloj: you lost the pre-condition ticket today... which buys you a little more time22:24
<whikloj>awoods: true, but if I have to keep rebase the pre-validate one I could be all week.22:25
<awoods>I'm out22:27
* whikloj leaves23:51

Generated by Sualtam