Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* thomz joins03:09
* ajs6f joins07:24
* dwilcox joins08:26
* thomz leaves09:09
* acoburn joins09:15
* dhlamb joins09:26
* peichman joins09:29
* osmandin joins09:33
* bseeger joins09:44
awoods: I don't know if you saw my irc message yesterday afternoon — I saw the random locking failure in the unit tests that acoburn was seeing: https://gist.github.com/bseeger/64d295bdc546fed2a77ee51675f28b4409:45
awoods: I was building acoburn's PR when it failed. Succeeded the second time I ran mvn clean install.09:46
* mikeAtUVa joins09:59
* osmandin leaves10:03
<awoods>bseeger: I got it, thanks.10:05
mikeAtUVa: I would suggest we get your patch into 4.7.0-RC-2 immediately for testing, and improve the integration tests as a separate ticket.10:06
* osmandin joins10:12
<acoburn>awoods: n.b.: they are unit tests, not integration tests that fail10:26
<ajs6f>awoods: Do we want to put the codebase into a non-deterministic state? Schrödinger's repo?
<awoods>acoburn/ajs6f: my point is that the issue appears to be with the tests, not the functionality. I do not propose putting out a release without working tests, but getting eyes on the functionality for release testing would be good. I have seen the swirl retooling unit/integration tests can get caught in.10:30
<ajs6f>awoods: I'm inclined to fix the unit tests _in the branch_. For the i-tests I'm more amenable to delay.10:31
<awoods>ajs6f: Meaning: "let's get the unit test working before release testing, the integration tests can come with time"?10:32
<acoburn>awoods: there is no failure w/ the integration tests
* mikeAtUVa leaves
<awoods>acoburn: got it
<acoburn>awoods: it's the unit tests that are problematic
<awoods>acoburn: got it
<ajs6f>awoods: Let's get the unit tests working before merging the branch.
<awoods>ajs6f: ok. Did you say you project having time next week?10:33
<ajs6f>awoods: For the i-tests, yes. Not for the u-tests.
<awoods>ajs6f: ok. Who is going to work the unit tests?10:34
<ajs6f>awoods; The person whose code is being tested.10:35
* osmandin leaves
<ajs6f>awoods: Nothing new there.
<awoods>ajs6f: does that person know that?
<ajs6f>awoods: I do not know. You should contact them, to be sure.10:36
awoods: I do not even know who the hell we are talking about. mikeAtUVA?
<awoods>ajs6f: That name sounds familiar.10:37
<ajs6f>awoods: Yes, you've met at conferences and that sort of thing. Rangy fellow with sandy reddish hair, dry sense of humor.
<awoods>To whom it may concern, In chatting with Stefano yesterday, the pairtree corruption was discovered in testing, not production.10:58
* osmandin joins11:05
<acoburn>awoods: related to the move to semantic versioning, will the fedora namespace URI drop the "v4" segment?11:18
<awoods>acoburn: good question. Having some element there to distinguish it from earlier (Fedora 3) versions likely makes sense. We could certainly change the "v4", although it does not seem to be absolutely necessary.11:20
* mikeAtUVa joins11:22
<acoburn>awoods: we have lots of time to discuss this, but changing the namespace to http://fedora.info/definitions/repository# seems to make sense to me
awoods: in principle, the NS could stay the same, but it may be confusing to some11:23
<awoods>acoburn: true... although changing the namespace can also be confusing.
* mikeAtUVa read the logs while he was gone... and it seemed to imply that I should fix unit tests. I am unable to get them to fail. bseeger seemed to report that they fail when building against another PR... has anyone had them fail under master?11:24
<acoburn>awoods: agreed
<awoods>mikeAtUVa: not I
<acoburn>mikeAtUVa: the transient failures occur on all branches that contain the locking commit11:25
mikeAtUVa: (for me)
mikeAtUVa: bseeger and I both use macs
<mikeAtUVa>acoburn do they contain commits after that commit? If so, which ones, so I can try those.
<acoburn>mikeAtUVa: I get failures on master, and on every branch that's been rebased to include your commit11:26
<mikeAtUVa>If the mac JVM interrupts threads of its own accord, I can imagine we'd have failures...
acoburn: it sounds like I wouldn't have trouble reproducing your errors if only I had a mac.11:27
<acoburn>mikeAtUVa: sometimes I get lucky and the tests pass on the first try, but usually it takes 2-5 times to get past the http-api module
<mikeAtUVa>acoburn: and by "your errors" I mean the errors in my code that you're seeing;)
ajs6f: you've got a mac, right?11:30
* github-ff joins11:33
[fcrepo4-vagrant] whikloj opened pull request #65: Use file-simple configuration for non-Auth setups (4.7.0-RC...FCREPO-2273) https://git.io/vPo56
* github-ff leaves
<osmandin>mikeAtUVa: I have a mac. Is the current build for master failing sometimes on mac?
* whikloj joins11:34
<mikeAtUVa>osmandin: yeah, the problem is likely entirely in one little testing construct... https://github.com/fcrepo4/fcrepo4/blob/master/fcrepo-http-api/src/test/java/org/fcrepo/http/api/DefaultPathLockManagerTest.java#L180-L236
<osmandin>mikeAtUVa: Thanks. I'm running it now.11:35
mikeAtUVa: Build success, the first time. Just FYI.11:37
<whikloj>awoods: ping
<awoods>whikloj11:38
* mikeAtUVa has go for a bit... I'll be back in 40 minutes...
* mikeAtUVa leaves11:39
<whikloj>awoods: two quick things, 1) do I need to cut a new RC with the patch?
<awoods>whikloj: yes, once there is a patch
<whikloj>awoods: 2) I figured out the 4.7.0 vagrant issue, https://jira.duraspace.org/browse/FCREPO-2273
awoods: sorry didn't mikeAtUVa submit a patch?
<awoods>whikloj: https://jira.duraspace.org/browse/FCREPO-227211:40
whikloj: is: https://jira.duraspace.org/browse/FCREPO-2273 a duplicate of: https://jira.duraspace.org/browse/FCREPO-2116 ?11:41
<whikloj>awoods: hmm yes perhaps it is, I'll fix that11:43
awoods: Can I close the duplicate or just link it as a duplicate and leave it open?11:44
<osmandin>mikeAtUVa: OK 2nd time as well. I'll try it on a different mac laptop or if I get time try to take a look.11:45
<awoods>whikloj: Yes, please close one of them as a duplicate.
osmandin: what do you mean by "2nd time as well"?
osmandin: sorry... I see now. It has passed two times.11:46
<osmandin>awoods: I ran "mvn clean install" again and got "build success"
* whikloj leaves
<acoburn>osmandin: for me it always fails on fcrepo-http-api, so in principle you only need to build that module
<awoods>acoburn/bseeger: are you running full builds? or "mvn clean install -pl fcrepo-http-api"?11:47
<acoburn>awoods: I do both
awoods: but on full builds they usually fail, so I have to use -rf :fcrepo-http-api
awoods: by "usually", I'm batting about 0.500
awoods: just tried it again on a clean master, failed 2/4 times11:48
* bseeger leaves11:51
<awoods>acoburn: It might be tricky for mikeAtUVa to resolve if he is not able to reproduce it locally.11:52
<acoburn>awoods: I agree. I also don't like transient errors that are only reproducible on certain platforms. It doesn't fill me with confidence11:53
<awoods>acoburn: agreed... clearly there is an issue here.11:54
<acoburn>awoods: it's not so different from my PR for FCREPO-1774 — I'm not able to reproduce the error that everyone else gets11:55
awoods: it makes me reconsider the implementation
* github-ff joins11:57
[fcrepo4-vagrant] awoods pushed 1 new commit to 4.7.0-RC: https://git.io/vPoNy
fcrepo4-vagrant/4.7.0-RC c325e30 Jared Whiklo: Use file-simple configuration for non-Auth setups (#65)
* github-ff leaves
<osmandin>mikeAtUVa: It failed for me for DefaultPathLockManagerTest (on a much older mac laptop) in http-api. Not sure if this is the error.11:59
DefaultPathLockManagerTest.deleteShouldBlockAccessToDescendants and writeShouldBlock12:00
<acoburn>osmandin: that's the error I encounter12:01
<osmandin>acoburn: Thanks. Are you passing any Xmx options, etc.?
<acoburn>osmandin: just this: MAVEN_OPTS="-Xmx1024m"12:02
osmandin: actually, let me verify that statement…12:03
osmandin: yes, that's the value I pass into maven12:04
<osmandin>acoburn: Thanks
<acoburn>osmandin: also, I'm using a MacBook Air from mid-2013
<osmandin>acoburn: Running again, I got an additional error for readShouldBlockWhileWriting. Just FYI.12:09
<acoburn>osmandin: yes, the failures are non-deterministic
osmandin: sometimes I get 1 failure, sometimes 3, and the specific failures depend on the particular run
* osmandin leaves12:13
* mikeAtUVa joins12:19
* ajs6f leaves12:32
* peichman1 joins12:57
<awoods>acoburn: are we waiting on https://jira.duraspace.org/browse/FCREPO-1931 due to conflicts with: https://jira.duraspace.org/browse/FCREPO-1868 ?12:58
* bseeger joins12:59
* peichman leaves
* dwilcox leaves13:04
* ajs6f joins13:05
<acoburn>awoods: I don't think so13:10
<awoods>acoburn: good. Will you be in a position today to move that PR forward?
<acoburn>awoods: possibly — I'm not entirely sure how I'd test it13:11
awoods: provision a VM with a very small amount of disk space?
awoods: and try to add a file that exceeds that amount?13:12
awoods: that would take some time to set up
<awoods>acoburn: that should certainly be one way. mikeAtUVa, are you able to easily test/reproduce: https://jira.duraspace.org/browse/FCREPO-1931 ?
<acoburn>awoods: ok, then probably not today13:13
<awoods>acoburn: understandable. There was some shuffling with that ticket and just wanted to make sure it was on someone's radar. Thanks.13:14
<acoburn>awoods: the code itself looks good
<awoods>acoburn: that is about where I stood.13:15
* youn joins13:17
<mikeAtUVa>awoods: I can test FCREPO-1931 if you'd like.13:18
<awoods>mikeAtUVa: that would be great. Please coordinate with acoburn.13:19
mikeAtUVa: what are your thoughts on the unit test for the 4.7.0 bug/patch?
<mikeAtUVa>awoods: there are some changes I'd try just looking at the code (perhaps the interrupt happens when it's not blocking on the mac and I should test for the interrupted flag rather than just caching the interrupted exception... maybe I'll make a really extensive change to try to get more information, have a mac user run it and work from there.13:21
<acoburn>mikeAtUVa: if you push changes to a public branch, I'll be happy to test them13:22
<ajs6f>mikeAtUVa: Can you give a link to the test in question?13:23
<mikeAtUVa>acoburn: thanks so much!
<ajs6f>mikeAtUVa: I'd just like to glance at it.
<mikeAtUVa>ajs6f: sure... I think it's all the ones where we're testing for blocking (invoking isBlocked() here https://github.com/fcrepo4/fcrepo4/blob/master/fcrepo-http-api/src/test/java/org/fcrepo/http/api/DefaultPathLockManagerTest.java#L211)13:24
<ajs6f>mikeAtUVa: The only way that Actor thing is going to block is on https://github.com/fcrepo4/fcrepo4/blob/master/fcrepo-http-api/src/test/java/org/fcrepo/http/api/DefaultPathLockManagerTest.java#L195, right?13:35
* ajs6f leaves13:39
* mikeAtUVa leaves13:42
<bseeger>mikeAtUVa: are unit tests run in any specific order,or is order unknown?13:44
mikeAtUVa: this test doesn't release the AcquiredLock (https://github.com/fcrepo4/fcrepo4/blob/master/fcrepo-http-api/src/test/java/org/fcrepo/http/api/DefaultPathLockManagerTest.java#L146) - however the test might get cleaned up in between - I'm not sure how java operates in that regard. The one thought I have is that it's using the same path as the previous test.13:46
* peichman1 leaves14:02
* peichman joins14:05
* osmandin joins14:06
* ajs6f joins14:32
* mikeAtUVa joins14:35
* dwilcox joins14:39
* dwilcox leaves14:43
* dwilcox joins14:44
* dhlamb_ joins14:50
* dhlamb leaves14:51
* awoods leaves
* osmandin leaves14:57
* awoods joins14:58
* bseeger leaves15:01
* bseeger joins15:29
* dwilcox leaves15:33
<mikeAtUVa>bseeger: I don't know the order the tests are run... it's certainly something that could vary across platforms.... but except for the mocks, there's nothing shared between the various tests that could be affected by order.15:37
bseeger: if they were run in parallel the shared mocked member variable could be a problem, but I don't think they are.15:38
acoburn: when you get a chance could you e-mail me the output of "mvn clean verify -pl fcrepo-http-api -Dtest=DefaultPathLockManagerTest -Dfcrepo.log.http.api=TRACE -DskipITs" when it fails? md5wz at virginia.edu15:46
<acoburn>mikeAtUVa: using the current master branch?15:48
<mikeAtUVa>acoburn: yeah.
<acoburn>mikeAtUVa: I'm on it
<mikeAtUVa>acoburn: thank you so much. (it should only take 30 seconds to run)15:49
acoburn: oh, and I verified that https://github.com/fcrepo4/fcrepo4/pull/1122 works (sounds like you looked at the code, so between us it's ready to merge?)15:51
<acoburn>mikeAtUVa: yes
* github-ff joins16:16
[fcrepo4] mikedurbin opened pull request #1124: Fixed a bug that cause intermitted test failures. (master...FCREPO-2270) https://git.io/vPKuu
* github-ff leaves
<acoburn>awoods: thanks for the review!16:19
<mikeAtUVa>acoburn: thanks for the output, I'm pretty sure I found the culprit.16:20
<acoburn>mikeAtUVa: that's great news
mikeAtUVa: is that the PR ^^^16:21
<mikeAtUVa>acoburn: yeah.
<acoburn>mikeAtUVa: I'll give it a try
* github-ff joins16:22
[fcrepo4] mikedurbin pushed 1 new commit to master: https://git.io/vPKz8
fcrepo4/master c4712b6 Andrew Woods: Ensure rest api returns 507 response on No Space Left (#1122)...
* github-ff leaves
<acoburn>mikeAtUVa: this seems to work16:28
<mikeAtUVa>acoburn: I snuck another commit in there that isn't stricctly necessary but is the reason it wasn't trivial for those encountering the error to know what was wrong.
<acoburn>mikeAtUVa: ok, I'll try that, too16:29
mikeAtUVa: I'm running through a set of 20 builds and they've all passed so far
<mikeAtUVa>acoburn: thanks... sorry to double your review work.:(
<acoburn>mikeAtUVa: once that finishes, I'll try it with the second commit16:30
<ajs6f>mikeAtUVa:bseeger: Just for fun: https://github.com/junit-team/junit4/wiki/test-execution-order16:36
Used to be whatever reflection returned, but not since 4.11
I can't think of why anyone in their right mind would write tests that have to run in a certain order, unless they were really misusing the semantics of @Test16:37
<mikeAtUVa>ajs6f: yeah, I found that... before verifying that I used a different DefaultPathLockManager instance for each test.
<ajs6f>mikeAtUVa: I have no reason to suppose it has any relevance to your PR. I just think it's interesting info.
mikeAtUVa: I would really love to be able to turn on parallel test execution in Surefire.16:38
<acoburn>mikeAtUVa: tests are all passing
<ajs6f>mikeAtUVa: That would massively speed up builds.
<mikeAtUVa>ajs6f: there's a lot of efficiency to be gained that way... ie, you can test "create an object", "verify that it's triples are there", "test deleting that object" ... instead we have duplicate the set up for tests that require it.
<ajs6f>mikeAtUVa: Yeah, that's the misusing the @Test semantics.
mikeAtUVa: That's not what @Test means.16:39
@Test implies that there may be pre and post conditions, but those conditions are not recorded in other @Tests
They are recorded either inside that @Test, or in @Before and @Beforeclass and @After and @Afterclass and @Rules.16:40
<mikeAtUVa>ajs6f: so I can't put asserts in my @before and @after?
;)
* ajs6f releases steam from his ears
<bseeger>mikeAtUVa: goto's, too (can you do that in Java)?16:41
mikeAtUVa: not that that really makes sense in this scenario. Just seeing how much steam can come out of ajs6f's ears.
<ajs6f>mikeAtUVa: I know you're kidding, but I just spent literally _days_ working with Andy Seaborne to track down a problem with a multithousand line PR I sent for Jena that kept running into the peculiarities of Jena's inter-test dependencies
<mikeAtUVa>awoods, ajs6f: in the meeting we discussed writing new Integration tests that gurantee precise thread timing. No ticket was created, and thus it doesn't seem a prerequisite for the backporting.... is that intentional?
<ajs6f>bseeger: There's a lot of hot air inside me. Or at least I seem to emit a lot of it.16:42
mikeAtUVa: I don't know. awoods and acoburn figured something out.
* travis-ci joins16:43
fcrepo4/fcrepo4#4779 (master - c4712b6 : Andrew Woods): The build passed.
Change view : https://github.com/fcrepo4/fcrepo4/compare/2b41b3d3838c...c4712b64b765
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/167747895
* travis-ci leaves
<ajs6f>bseeger: https://docs.oracle.com/javase/specs/jls/se8/html/jls-14.html#jls-14.7
bseeger: That's as close as you get in vanilla Java.16:44
<awoods>mikeAtUVa: I believe ajs6f said he would rework your ITs next week... he had some issue with them. ajs6f, would you be willing to write a ticket?16:46
<ajs6f>awoods:mikeAtUVa: I would rather mikeAtUVa write the ticket.
<awoods>mikeAtUVa: if you know what the reservations were with your IT, could you write the ticket?16:47
<mikeAtUVa>ajs6f, awoods: I can do it. Are we waiting on ajs6f finishing that ticket before the backporting?
* ajs6f leaves
<awoods>mikeAtUVa: no, the unit tests were the blocker
<mikeAtUVa>awoods: Ok.16:48
awoods: I have slight concerns that the scrutiny required to write those tests are more likely to unearth possible bugs that any testing we could reasonably do. That said, we can always backport the test or the fix to any unearthed bugs.16:49
<awoods>mikeAtUVa: if you have a suspicion that there are lurking bugs, would you rather we wait on the IT?
<acoburn>awoods: mind if I merge mikeAtUVa's last PR?16:51
<awoods>acoburn: hit it
<mikeAtUVa>awoods: I don't think there are lurking bugs.
<awoods>mikeAtUVa: then what were you suggesting?
* github-ff joins
[fcrepo4] acoburn pushed 1 new commit to master: https://git.io/vPKVW
fcrepo4/master 32f4e4e Michael Durbin: Fixed a bug that cause intermitted test failures. (#1124)...
* github-ff leaves
<mikeAtUVa>awoods: I don't think we should wait on those tests... I guess I mean something more along the lines of "it's entirely possible that there are bugs or shortcomings, though I don't think critical ones are likely".16:55
* youn leaves16:56
* bseeger leaves16:57
<awoods>mikeAtUVa++16:58
* travis-ci joins17:15
fcrepo4/fcrepo4#4781 (master - 32f4e4e : Michael Durbin): The build passed.
Change view : https://github.com/fcrepo4/fcrepo4/compare/c4712b64b765...32f4e4e070f2
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/167754849
* travis-ci leaves
<acoburn>awoods: https://github.com/fcrepo4-labs/fcrepo-vocabulary17:17
awoods: it uses commons-rdf
awoods: if this passes muster, I'd like this to replace the current RdfLexicon that we've got in Fedora17:18
awoods: re https://jira.duraspace.org/browse/FCREPO-2270 should this be re-opened as part of the back-port to 4.6.0?17:20
awoods: nevermind, I see there's a separate ticket for that
* acoburn leaves17:34
* peichman leaves17:38
* youn joins18:49
* youn leaves18:54
* mikeAtUVa leaves21:13
* mikeAtUVa joins21:25
* github-ff joins21:39
[fcrepo4] mikedurbin opened pull request #1125: Back-ported two commits. (4.7.0-RC...FCREPO-2272) https://git.io/vPKhV
* github-ff leaves
* github-ff joins22:56
[fcrepo4] mikedurbin opened pull request #1126: Back-ported two commits. (4.6.0-maintenance...FCREPO-2271) https://git.io/vP6vw
* github-ff leaves
* f4jenkins joins23:13

Generated by Sualtam