Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* nbanks joins06:42
* github-ff joins07:59
[fcrepo-jms-indexer-pluggable] yecao closed pull request #6: adopt Spring constructor injection (master...spring_constructor_injection_adopt) http://git.io/wrU1Rw
* github-ff leaves
* ksclarke joins08:46
* github-ff joins08:57
[fcrepo-fedora3-federation-connector] awoods pushed 1 new commit to master: http://git.io/DMbE9w
fcrepo-fedora3-federation-connector/master c3bbf58 Andrew Woods: Update version dependency on local-legacy-fedora3
* github-ff leaves
* kaarefc leaves09:03
* travis-ci joins09:05
[travis-ci] futures/fcrepo-fedora3-federation-connector#22 (master - c3bbf58 : Andrew Woods): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo-fedora3-federation-connector/compare/652881a4b04f...c3bbf58c88b9
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo-fedora3-federation-connector/builds/12416487
* travis-ci leaves
* fasseg joins09:14
hey guys!
* gregjansen joins09:20
<fasseg>awoods: I'll continue to work on the SCC cluster then: https://www.pivotaltracker.com/story/show/5228954509:21
<pivotal-bot_____>feature: Run benchmarks on SCC cluster at FIZ Karlsruhe with fcrepo3 and fcrepo4 (unscheduled) / owner: Frank Asseg
* github-ff joins09:22
[fcrepo-jms-indexer-pluggable] yecao opened pull request #8: adopt constructor-arg for solrServerFactory and solrIndexer in indexer-c... (master...spring_injection_refactory) http://git.io/zyspNA
* github-ff leaves
<awoods>fasseg: Welcome back. I will catch you in about 1 hour
* github-ff joins09:24
[fcrepo-fedora3-federation-connector] awoods pushed 2 new commits to master: http://git.io/TldK7A
fcrepo-fedora3-federation-connector/master 272e4cb Andrew Woods: [maven-release-plugin] prepare release fcrepo-fedora3-federation-4.0.0-alpha-2
fcrepo-fedora3-federation-connector/master 14eb454 Andrew Woods: Update project_name to reflect github repository name
* github-ff leaves
<fasseg>awoods: k
* kaarefc joins09:25
<bljenkins>Project fcrepo-fedora3-federation-connector build #45: FAILURE in 2 min 12 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fedora3-federation-connector/45/09:26
* awoods: [maven-release-plugin] prepare release fcrepo-fedora3-federation-4.0.0-alpha-2
* awoods: Update project_name to reflect github repository name
* travis-ci joins09:29
[travis-ci] futures/fcrepo-fedora3-federation-connector#23 (master - 14eb454 : Andrew Woods): The build has errored.
[travis-ci] Change view : https://github.com/futures/fcrepo-fedora3-federation-connector/compare/c3bbf58c88b9...14eb454aff35
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo-fedora3-federation-connector/builds/12417628
* travis-ci leaves
* osmandin joins09:30
* github-ff joins09:38
[fcrepo-fedora3-federation-connector] awoods pushed 1 new commit to master: http://git.io/Qtjc5g
fcrepo-fedora3-federation-connector/master 20b677e Andrew Woods: Revert previous [maven-release-plugin] commit
* github-ff leaves
<bljenkins>Yippie, build fixed!09:44
Project fcrepo-fedora3-federation-connector build #46: FIXED in 6 min 8 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fedora3-federation-connector/46/
awoods: Revert previous [maven-release-plugin] commit
* travis-ci joins
[travis-ci] futures/fcrepo-fedora3-federation-connector#24 (master - 20b677e : Andrew Woods): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo-fedora3-federation-connector/compare/14eb454aff35...20b677e9fc21
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo-fedora3-federation-connector/builds/12418199
* travis-ci leaves
* fasseg leaves09:49
* fasseg joins
* kaarefc leaves09:56
* github-ff joins10:02
[fcrepo-fedora3-federation-connector] awoods pushed 2 new commits to master: http://git.io/eDVexg
fcrepo-fedora3-federation-connector/master 0b3a3e7 Andrew Woods: [maven-release-plugin] prepare release fcrepo-fedora3-federation-4.0.0-alpha-2
fcrepo-fedora3-federation-connector/master 314e8dc Andrew Woods: [maven-release-plugin] prepare for next development iteration
* github-ff leaves
<pivotal-bot_____>Mike Durbin started "Create a RepositoryOrganizer implementation to present objects in a fedora 3 federation into reasonably sized groups." https://www.pivotaltracker.com/story/show/5811587010:03
* travis-ci joins10:04
[travis-ci] futures/fcrepo-fedora3-federation-connector#25 (master - 314e8dc : Andrew Woods): The build has errored.
[travis-ci] Change view : https://github.com/futures/fcrepo-fedora3-federation-connector/compare/20b677e9fc21...314e8dce569f
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo-fedora3-federation-connector/builds/12419539
* travis-ci leaves
* gregjansen leaves10:07
* github-ff joins10:21
[fcrepo-fedora3-federation-connector] awoods created release-alpha-2 from master (+0 new commits): http://git.io/T6UIFQ
* github-ff leaves
* github-ff joins10:25
[fcrepo-fedora3-federation-connector] awoods tagged fcrepo-fedora3-federation-4.0.0-alpha-2 at 203c0f9: http://git.io/iQGfYQ
* github-ff leaves
* github-ff joins10:32
[fcrepo-fedora3-federation-connector] awoods created gh-pages (+1 new commit): http://git.io/ljfRNw
fcrepo-fedora3-federation-connector/gh-pages e8e634b Andrew Woods: Creating site for fcrepo-fedora3-federation,...
* github-ff leaves
<fasseg>yep10:33
<barmintor>fasseg: nice to see you online again. How was your holiday?10:40
<fasseg>yeah thanks, nice to be back on fedora, south east asia was great, and I shall greet you all from eddie...and we finished the whiskey you guys gave him ;)10:44
<awoods>ajs6f: standup?11:02
* github-ff joins11:18
[fcrepo-fedora3-federation-connector] mikedurbin opened pull request #9: Created a RepositoryOrganizer that ensures configurable group sizes. (master...58115870-hierarchy-2) http://git.io/0A-7Cg
* github-ff leaves
<pivotal-bot_____>Mike Durbin added comment: "Addressed with pull request: https://github.com/futures/fcrepo-fedora3-federation-connector/pull/9
" https://www.pivotaltracker.com/story/show/58115870
Mike Durbin finished "Create a RepositoryOrganizer implementation to present objects in a fedora 3 federation into reasonably sized groups." https://www.pivotaltracker.com/story/show/5811587011:19
* ajs6f joins11:30
awoods: I just missed standup, didn't I?11:31
Sorry.
<awoods>ajs6f: pretty much
<ajs6f>I was in a meeting and lost track of time. I apologize.
Here's my standup:
Slow, steady progress on the "iterate all the RDFs" front.
<awoods>ajs6f: np
<ajs6f>It turns out that all of our generating apparatus for this stuff is interlinked. For example, in order to11:32
rework the generation of "structural" RDF, I need to rework the generation of "data property" RDF.
Or I could unlink the generators, but I'm trying to work as much "in place" as possible.11:33
And I have no idea who or what is calling these things from outside the kernel.
(No clear contracts for the kernel… we could talk about that in Austin.)
The tests are >1000 lines of mocking, much of which is c&p. I think I can blame myself for that.11:34
So that needs to be factored a little so I can keep my sanity while I do this work.
BUt overall, slow steady progress. I hope to have a commit or two shortly.11:35
<awoods>on a call
* jongibson joins11:51
<awoods>ajs6f: Thanks for the update. It sounds like you are facing some issues with the structure of the code. There is currently an Austin topic of discussing the kernel-RDF contracts. It may also be valuable to consider what refactoring would be constructive going forward as well.12:14
* osmandin leaves12:17
* ajs6f leaves12:18
* github-ff joins12:20
[fcrepo-fedora3-federation-connector] awoods pushed 1 new commit to gh-pages: http://git.io/uLT9zw
fcrepo-fedora3-federation-connector/gh-pages 8e859ad Andrew Woods: Creating site for fcrepo-fedora3-federation,...
* github-ff leaves
* ajs6f joins12:45
awoods: ping?
<awoods>hi
<ajs6f>awoods; just a note on the way: I just noticed that our low level gear to return persistence metadata is also transacting in in-memory constructs (Sets).12:46
IOW, the same problem as with the RDF will arise here.
A big datastream will be chunked by IPSN.
ISPN.
And might end up as thousands of chunks.
Info of which will be passed through the heap, not iterated.12:47
(Incidentally, we're reporting this stuff in the RDF< as well.)
<awoods>ajs6f: What is the "persistence metadata" here?12:48
<ajs6f>Information about the chunks ISPN is storing. Where they are, what their keys are, etc.
The chunk size is set by ISPN. We can't control it.12:49
<awoods>Is there an iterable option?
<ajs6f>Dunno. I hope so.
cbeer/barmintor: do you remember?
<awoods>These are just pointers to chunks, I assume
<ajs6f>Yes, but there will be _a lot_ of them for large datastreams.12:50
And then consider that this is info scoped to a single request. Could be many requests. It could add up quickly.
<awoods>Given an ISPN-defined chunksize, I wonder how large of a datastream will create an issue.
<ajs6f>ISPN (IIRC) chunks at 1MB.
<cbeer>s/ISPN/MODE/, no?12:51
<ajs6f>5 GB / 1MB = 5000. 5000 times a couple of triples.
cbeer: I don't remember. If you say so, I believe you. But it's out of our control, anyway, right?
<cbeer>and why are we exposing that as triples now?
<ajs6f>You tell me. I didn't do it.12:52
If an object has a datastream,
<cbeer>i don't remember anything about individual ISPN chunks
<ajs6f>those triples get generated for a request on that object's basic presentation.
<barmintor>ajs6f: I am totally out of context here. Not much to contribute w/o catch-up.
<ajs6f>Hang on, I'll find it for you.
barmintor: Fair enough. Go back to fixing fcrepo3. {grin}
<barmintor>yeah, basically.12:53
<ajs6f>cbeer: https://github.com/futures/fcrepo4/blob/master/fcrepo-kernel/src/main/java/org/fcrepo/kernel/utils/JcrRdfTools.java#L66112:55
llstore.getLowLevelCacheEntries()
<cbeer>ajs6f: that's not serializing ISPN chunks, is it?
<ajs6f>I think it is. What else is it doing?
<cbeer>ajs6f: i think it re-constitutes the original byte streams by asking each ISPN node if it has the thing12:56
(which isn't great in a distribution scenario..)
<ajs6f>Urg. Wait, so in order to provide RDF, we're _building the original bytestreams_?12:57
<cbeer>maybe something's changed
ajs6f: no, no. that's just for doing fixity or whatever
getLowLevelCacheEntries is metadata only
wherever it is.
<ajs6f>Metadata on what? On the different caches? On the chunks?
<cbeer>you'd have to look at the MODE internals, but i don't think it stores metadata on each 1MB chunk12:58
i think it's just a count of how many chunks there are12:59
<ajs6f>Okay. But we're really not sure about what our own LowLevelCacheEntry class actually _is_? (Not that I have been at all helpful with any of that.)
<cbeer>ajs6f: i'd say generally, don't worry about it. although the TODO makes me wonder.
<ajs6f>Oh, np. I wasn't going to worry. I asked awoods to do that.
<cbeer>ajs6f: it used to be a way to get at the full binary blob13:00
<ajs6f>I'm busy being mocked by mocks. JcrRdfTools is a thousand-line class with a thousand lines of mocking to test it. {grumble}
<cbeer>i don't know if that's still true.. but it'd surprise me if any of you had the time to refactor that since july
<ajs6f>I don't think anyone has touched that stuff.
<awoods>cbeer/ajs6f: correct, it has not been touched
<ajs6f>Okay, back to the refactoring mines. I'm just glad I'm not barmintor.13:01
<barmintor>so am… wait a second!
<ajs6f>Hurts, don't it.13:02
Good thing you're coming home to an incredible beer experience.
<barmintor>I need to book a ticket.
<ajs6f>To Austin?
<barmintor>yeah.13:03
<ajs6f>Yeah, I'd do that.
<barmintor>wait, were you talking about my home home? Hmm. I am going to have a superlative beer experience tomorrow, but it's going to be generally credible.
<ajs6f>barmintor: If you told me you had recently drunk beer made on a moonlit night from the tears of Belgian virgins with hops grown in space, I would find it credible.13:04
<barmintor>You heard about Tuesday?13:05
<ajs6f>I'm assuming that awoods will bring some to Austin!
<awoods>I can bring some Silver Bullet. ;)13:06
<ajs6f>Lone Star, buddy. Lone Star.
Actually, I encourage everyone coming to try the Live Oak Pilz. Really delicious.13:08
<barmintor>Live Oak is good stuff, but I'm going to be foisting other TX beers on you saps.13:10
<ajs6f>This sap is ready to rise. To the occasion. Oh, you know what I mean.13:11
* github-ff joins13:18
[fcrepo4] ajs6f pushed 1 new commit to RDFIteration: http://git.io/BNQq_Q
fcrepo4/RDFIteration 32b2c32 ajs6f: Beginning refactoring of the blizzard of mocking laughter that is JcrRdfToolsTest
* github-ff leaves
* osmandin joins13:28
* github-ff joins
[fcrepo4] ajs6f pushed 1 new commit to RDFIteration: http://git.io/ku4TvA
fcrepo4/RDFIteration df5b5ca ajs6f: More cleanup in JcrRdfToolsTest
* github-ff leaves
<ajs6f>I'm leaving the office for the day. I'll check in later.13:29
* ajs6f leaves
* github-ff joins13:34
[fcrepo4] ajs6f pushed 1 new commit to RDFIteration: http://git.io/DJwLJA
fcrepo4/RDFIteration 25724b4 ajs6f: Adding some logging and new tests
* github-ff leaves
<pivotal-bot_____>Andrew Woods added "Intermittent test errors: AtomJMSIT " https://www.pivotaltracker.com/story/show/5872651213:43
Andrew Woods started "Intermittent test errors: AtomJMSIT " https://www.pivotaltracker.com/story/show/58726512
* github-ff joins14:19
[fcrepo4] awoods pushed 1 new commit to master: http://git.io/0EmEEw
fcrepo4/master 9e7a174 Andrew Woods: Minor: Ensure AtomJMSIT starts with consistent state...
* github-ff leaves
<pivotal-bot_____>Andrew Woods added comment: "Resolved with: http://git.io/0EmEEw" https://www.pivotaltracker.com/story/show/58726512
Andrew Woods delivered "Intermittent test errors: AtomJMSIT " https://www.pivotaltracker.com/story/show/58726512
* travis-ci joins14:33
[travis-ci] futures/fcrepo4#1011 (master - 9e7a174 : Andrew Woods): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/acc3f4fd3c74...9e7a174697d4
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/12430783
* travis-ci leaves
* jongibson leaves14:46
* jongibson joins14:57
<bljenkins>Yippie, build fixed!15:12
Project fcrepo4 build #1170: FIXED in 25 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1170/
<pivotal-bot_____>Osman Din added comment: "Initial commit to evaluate settings for Infinispan LevelDb: ""15:42
https://github.com/osmandin/fcrepo4/commit/a6b035c..." https://www.pivotaltracker.com/story/show/57835034
<awoods>afk15:44
* ajs6f joins16:05
* fcrepo-bot joins16:22
<pivotal-bot_____>Osman Din added comment: "Initial commit to test ispn LevelDb settings, but fcrepo-webhooks tests fail: https://github.com/osmandin/fcre..." https://www.pivotaltracker.com/story/show/5783503416:38
* nbanks leaves16:43
* nbanks joins
* osmandin leaves16:53
* mikeAtUVa leaves16:56
* barmintor leaves17:04
* nbanks leaves17:10
* fcrepo-bot leaves17:26
* jongibson leaves17:37
* jongibson joins18:06
* fasseg leaves18:08
* nbanks joins18:30
* github-ff joins18:33
[fcrepo4] awoods pushed 1 new commit to master: http://git.io/IYP_jw
fcrepo4/master b673667 Andrew Woods: Address intermittent AtomJMSIT test errors...
* github-ff leaves
* nbanks leaves
<pivotal-bot_____>Andrew Woods added comment: "...and resolved with: http://git.io/IYP_jw" https://www.pivotaltracker.com/story/show/58726512
* travis-ci joins18:45
[travis-ci] futures/fcrepo4#1012 (master - b673667 : Andrew Woods): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/9e7a174697d4...b673667380c0
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/12440782
* travis-ci leaves
* ajs6f leaves
<bljenkins>Project fcrepo4 build #1171: UNSTABLE in 16 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1171/18:50
awoods: Address intermittent AtomJMSIT test errors
<pivotal-bot_____>Andrew Woods unstarted "Intermittent test errors: AtomJMSIT " https://www.pivotaltracker.com/story/show/5872651219:39
Andrew Woods edited "Intermittent test errors: AtomJMSIT " https://www.pivotaltracker.com/story/show/58726512
Andrew Woods edited "Intermittent test errors: AtomJMSIT " https://www.pivotaltracker.com/story/show/58726512
Andrew Woods edited "Intermittent test errors: AtomJMSIT " https://www.pivotaltracker.com/story/show/5872651219:40
* jongibson leaves20:07
* github-ff joins20:11
[fcrepo-legacy-api] awoods pushed 1 new commit to master: http://git.io/ot3_jw
fcrepo-legacy-api/master 2082616 Andrew Woods: Update to alpha-3-snapshot
* github-ff leaves
* github-ff joins20:50
[fcrepo-bagit-object-serialization] awoods pushed 1 new commit to master: http://git.io/45L4tQ
fcrepo-bagit-object-serialization/master 1ddc10b Andrew Woods: Update to alpha-3-snapshot
* github-ff leaves
<bljenkins>Project fcrepo-bagit-object-serialization build #25: SUCCESS in 4 min 51 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-bagit-object-serialization/25/20:55
Project fcrepo-sequencer-archives build #22: SUCCESS in 5 min 29 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-sequencer-archives/22/21:35
* github-ff joins21:36
[fcrepo-kitchen-sink] awoods pushed 1 new commit to master: http://git.io/VtyHqA
fcrepo-kitchen-sink/master e045670 Andrew Woods: Update to alpha-3-snapshot
* github-ff leaves
<bljenkins>Project fcrepo-kitchen-sink build #533: FAILURE in 2 min 0 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/533/21:37
Project fcrepo-fixity-corrupter build #277: SUCCESS in 4 min 5 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/277/22:33
Project fcrepo-kitchen-sink build #534: STILL FAILING in 9 min 43 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/534/22:40
Yippie, build fixed!22:53
Project fcrepo-kitchen-sink build #535: FIXED in 13 min: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/535/