Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* ksclarke leaves00:09
* kaarefc joins01:40
* kaarefc leaves01:51
* kaarefc joins01:55
* kaarefc leaves02:05
* kaarefc joins02:32
* fasseg joins03:28
* leenata joins03:43
<fasseg>morning leenata03:44
<leenata>morgen03:46
<fasseg>eddies: btw formatting using java-formatter-plugin was an exercise in futility, it seems the plugin does use tabs *always* even if I the eclipse configuration used is set to use spaces instead of tabs :/
<eddies>hrm
ah well
<fasseg>yeah soll ill create a checkstyle config, that we emit warnings at least when the formatting doesnt fit.....03:47
eddies: performance wise, there are no more easy targets it seems, after the configuration tweaks and the DefaultFilter patch. All I see now are FileChannels operations from Infinispan and the lucene operations on modeshape, that cost a considerable amount of time...03:58
Im able to reach about 160ms of ingest time with query enabled, but I still feel that's bad enough to not use JQL ina default install ;)04:02
so real ingest times with binary data, not just fcr:create04:03
and if textfiles would be ingested and lucene also has to do a full text indexing on those, I fear it'll become too slow to work with04:04
BTW: Ill steal from the Presentations on the wiki for a short intro talk for SCAPE tomorrow and for FIZ Karlsruhe next Tuesday...04:20
<eddies>actually, can you two get on hangouts right now for a call?04:21
<fasseg>I can..
<eddies>leenata?
http://bugs.sun.com/view_bug.do?bug_id=470509304:45
http://stackoverflow.com/questions/137212/how-to-solve-performance-problem-with-java-securerandom04:46
<fasseg>http://stackoverflow.com/questions/137212/how-to-solve-performance-problem-with-java-securerandom
<eddies>-Djava.security.egd=file:/dev/./urandom
<fasseg>http://commons.apache.org/proper/commons-lang/javadocs/api-2.6/org/apache/commons/lang/math/RandomUtils.html04:49
* bljenkins leaves04:51
<fasseg>http://opennebula.org/about:about04:58
* bljenkins joins04:59
<fasseg> https://www.pivotaltracker.com/story/show/5156671305:09
<pivotal-bot_>feature: Add unit test coverage to FedoraVersions (unstarted) / owner: Chris Beer
Edwin Shin edited "Add unit test coverage to FedoraVersions" https://www.pivotaltracker.com/story/show/5156671305:12
Edwin Shin edited "Add test coverage to FedoraResource" https://www.pivotaltracker.com/story/show/5156594305:13
<leenata>hey eddies05:22
will start this
<eddies>leenata: yes, please start on #51566713 first05:26
<leenata>cool05:28
<fasseg>leenata: and start the ticket, or someone might grab it from you again ;)05:29
<leenata>grabbed :)05:30
<fasseg>hmm I mean click on "start" in the ticket, so other pivotal users see that you already working on that ticket...05:31
or did it just not update for me?05:32
<leenata>ya thats what i did. now is it showing?05:34
<fasseg>hmm still "unstarted" for me05:35
eddies: Ill create a ticket for the clustered 1TB test using JMeter with fcrepo3 & fcrepo405:36
<eddies>danke
<pivotal-bot_>Frank Asseg added "Run benchmarks on SCC cluster at FIZ Karlsruhe with fcrepo3 and fcrepo4" https://www.pivotaltracker.com/story/show/5228954505:40
Leenata started "Add unit test coverage to FedoraVersions" https://www.pivotaltracker.com/story/show/51566713
<leenata>my fault! missed the save button above
<pivotal-bot_>Edwin Shin added "Eclipse lifecycle-mapping plugin errors" https://www.pivotaltracker.com/story/show/5229092106:11
Edwin Shin started "Eclipse lifecycle-mapping plugin errors" https://www.pivotaltracker.com/story/show/52290921
Edwin Shin added comment: "Created https://github.com/futures/dummy-lifecycle-mapping-plugin and deployed to fcrepo-thirdparty" https://www.pivotaltracker.com/story/show/5229092106:12
Edwin Shin accepted "Eclipse lifecycle-mapping plugin errors" https://www.pivotaltracker.com/story/show/5229092106:13
<fasseg>eddies: I just saw that Im not on the sprint schedule for sprint13.....I'll just change that or will you kick me out of the team? :)07:42
<eddies>heh. go ahead and put yourself down.07:45
<fasseg>you're not in their either btw: https://wiki.duraspace.org/display/FF/Sprint+Schedule07:52
*there
* bljenkins leaves08:12
* bljenkins joins08:13
* jcoyne joins08:17
* jcoyne leaves08:36
* ksclarke joins09:39
* github-ff joins09:47
[fcrepo4] eddies pushed 1 new commit to master: http://git.io/3ngFHw
fcrepo4/master 9f0074a Edwin Shin: Checkstyle:...
* github-ff leaves
<cbeer>eddies: can you figure out how to raise the checkstyle linelength10:09
(to 120? i can't remember if we made a decision on that.)
<awoods>seemed like the compromise was 96
<bljenkins>Project fcrepo-fixity build #353: FAILURE in 1 min 37 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/353/10:14
* kaarefc leaves10:15
* bljenkins leaves10:18
* bljenkins joins10:19
Project fcrepo-fixity-corrupter build #168: FAILURE in 7 min 50 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity-corrupter/168/10:20
* jcoyne joins10:22
<bljenkins>Project fcrepo-kitchen-sink build #426: FAILURE in 9 min 50 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-kitchen-sink/426/
* leenata leaves10:44
<eddies>cbeer: you're suddenly quiet10:47
<cbeer>:/10:50
* travis-ci joins10:52
[travis-ci] futures/fcrepo4#773 (master - 9f0074a : Edwin Shin): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/9beffaeb38ce...9f0074a060e3
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8425312
* travis-ci leaves
<cbeer>awoods: https://www.pivotaltracker.com/story/show/4817000311:03
<pivotal-bot_>feature: Replace state transfer with ClusterCacheLoader? (unstarted) / owner:
<fasseg>sry I missed the hangout but I was stuck in traffic after getting my GF from the hospital....11:11
but sprint close in 50 mins right?
<cbeer>fasseg: yes11:12
awoods: ok, raised the q with rhauch.. seems like we might have some work in the next phase if we want to tackle it11:13
<pivotal-bot_>Chris Beer added comment: "https://github.com/futures/fcrepo4/blob/master/fcrepo-http-api/src/test/java/org/fcrepo/integration/api/Fedor..." https://www.pivotaltracker.com/story/show/4901238912:05
Edwin Shin accepted "Ensure REST API supports workspace identifier prefixes by adding tests" https://www.pivotaltracker.com/story/show/4901238912:06
Edwin Shin accepted "alter FedoraTransactions so that it no longer maintains the state of the transaction store inside the endpoint itself" https://www.pivotaltracker.com/story/show/5160251312:07
Edwin Shin accepted "Add pagination for child node listings in rdf responses" https://www.pivotaltracker.com/story/show/51183927
Chris Beer added comment: " curl -I http://fcrepo4.fcrepo.org/fcrepo/rest//fcr:export?format=bagit12:08
HTTP/1.1 200 OK
Date: Tue, 25 Jun 201..." https://www.pivotaltracker.com/story/show/46839587
Edwin Shin accepted "Make fcrepo-object-serialization handle mime-types" https://www.pivotaltracker.com/story/show/4683958712:09
Edwin Shin accepted "Make existing REST API transactions-aware" https://www.pivotaltracker.com/story/show/49010571
Edwin Shin accepted "Document recommended heap and stack sizes for running fcrepo4" https://www.pivotaltracker.com/story/show/5156913912:10
Edwin Shin accepted "Add unit test coverage to uri resource graph injection classes" https://www.pivotaltracker.com/story/show/51566849
Edwin Shin accepted "Wire in a JAX-RS resource to manipulate JCR workspaces" https://www.pivotaltracker.com/story/show/51675235
Edwin Shin accepted "Within a transaction, graph subjects should include the transaction identifier for all node-subjects." https://www.pivotaltracker.com/story/show/5171751312:12
Chris Beer added "Add TX information to the graph response, or introspect the URL for a TX identifier and add the appropriate actions gl..." https://www.pivotaltracker.com/story/show/52316287
Edwin Shin accepted "java.lang.NullPointerException at org.fcrepo.utils.impl.CacheStoreEntry.getExternalIdentifier(CacheStoreEntry.java:87)" https://www.pivotaltracker.com/story/show/5181245712:13
Edwin Shin accepted "Run fcrepo4 single-node, async indexing configuration performance tests" https://www.pivotaltracker.com/story/show/51516221
Edwin Shin accepted "Run fcrepo4 single-node configuration performance tests" https://www.pivotaltracker.com/story/show/51516137
Edwin Shin accepted "Give HTML responses a coat of bootstrap styling." https://www.pivotaltracker.com/story/show/52018343
Edwin Shin added comment: "https://wiki.duraspace.org/display/FF/Fedora+4+test+results+-+AWS+cluster+of+four++-+2013-06-25" https://www.pivotaltracker.com/story/show/5156718712:15
Edwin Shin accepted "Enable auto-clustering of servers deployed with Puppet" https://www.pivotaltracker.com/story/show/51567187
Edwin Shin accepted "Make FedoraTransactions more RESTful" https://www.pivotaltracker.com/story/show/51680545
Edwin Shin accepted "Add transactions, sparql-update, and namespace actions to the text/html responses" https://www.pivotaltracker.com/story/show/5210323112:16
Chris Beer added "Add import format triples to every node graph (or get that information from the repository resource somehow..)" https://www.pivotaltracker.com/story/show/5231687112:19
Edwin Shin accepted "Add import action to the text/html response" https://www.pivotaltracker.com/story/show/52130575
* bljenkins leaves
* cbeer leaves
* cbeer joins
* bljenkins joins12:20
<pivotal-bot_>Edwin Shin added comment: "No longer using fcrepo-client, just parsing responses directly w/ Jena" https://www.pivotaltracker.com/story/show/5044821512:23
Edwin Shin accepted "Update fcrepo-fixity service to use new RDF responses" https://www.pivotaltracker.com/story/show/50448215
<cbeer>fcrepo4: if it's not working, people aren't going to use it.12:24
<pivotal-bot_>Edwin Shin accepted "Fixity webapp cleanup" https://www.pivotaltracker.com/story/show/50879139
Edwin Shin accepted "Publish JMeter reports to Sonar via Maven" https://www.pivotaltracker.com/story/show/5151775312:25
<eddies>https://github.com/futures/fcrepo4/pull/8612:26
<pivotal-bot_>Edwin Shin accepted "Run fcrepo3 performance tests on AWS" https://www.pivotaltracker.com/story/show/5151598712:35
Edwin Shin finished "Sprint 12 validation of fcrepo4 on Windows" https://www.pivotaltracker.com/story/show/5151791512:36
Edwin Shin accepted "Sprint 12 validation of fcrepo4 on Windows" https://www.pivotaltracker.com/story/show/51517915
Edwin Shin delivered "Sprint 12 validation of fcrepo4 on Windows" https://www.pivotaltracker.com/story/show/51517915
Frank Asseg added comment: "added test framework with this commit: ""12:38
https://github.com/futures/fcrepo-fixity/commit/1a892a233c6c5a3150fb8..." https://www.pivotaltracker.com/story/show/50647767
Frank Asseg added comment: "cleup in this commit: https://github.com/futures/fcrepo-fixity/commit/1015704787e76c96d505cc9df374c5697150217e" https://www.pivotaltracker.com/story/show/5064776712:40
<cbeer>https://www.pivotaltracker.com/reports/point_count/project?iteration_designation=current&project_id=684825&chart_type=project&commit=Generate+Report
<pivotal-bot_>Frank Asseg added comment: "added unit tests: https://github.com/futures/fcrepo-fixity/commit/6799e02a55fecd2149b8480f7d6f0d97ce015119" https://www.pivotaltracker.com/story/show/50647767
Frank Asseg accepted "Improve testability of fixity webapp" https://www.pivotaltracker.com/story/show/5064776712:42
Frank Asseg added comment: "added configuration with this commit: https://github.com/futures/fcrepo4/commit/6539df675c67515e7791a15f8fb2..." https://www.pivotaltracker.com/story/show/5196345712:43
<fasseg>eddies: added commit comment to https://www.pivotaltracker.com/story/show/51963457 so you can accept the ticket...
<pivotal-bot_>feature: Add configuration without indexing and fSyncMode="default" for max ingest performance (finished) / owner: Frank Asseg
<cbeer>https://docs.jboss.org/author/display/ISPN/Contributing+to+Infinispan12:56
brb12:57
back13:00
* eddies1 joins13:14
* eddies1 leaves
* eddies1 joins
* eddies leaves13:21
<pivotal-bot_>Chris Beer added "Run profiling against current fcrepo4 master and look for any easy bottlenecks to clear off" https://www.pivotaltracker.com/story/show/5232565913:52
Chris Beer edited "Run profiling against current fcrepo4 master and look for any easy bottlenecks to clear off" https://www.pivotaltracker.com/story/show/52325659
Chris Beer edited "Run profiling against current fcrepo4 master and look for any easy bottlenecks to clear off" https://www.pivotaltracker.com/story/show/52325659
Chris Beer edited "Add import format triples to every node graph (or get that information from the repository resource somehow..)" https://www.pivotaltracker.com/story/show/52316871
Chris Beer edited "Add TX information to the graph response, or introspect the URL for a TX identifier and add the appropriate actions globally." https://www.pivotaltracker.com/story/show/52316287
Chris Beer edited "Test fcrepo4 with a large random data set" https://www.pivotaltracker.com/story/show/52213823
Chris Beer edited "Run benchmarks on SCC cluster at FIZ Karlsruhe with fcrepo3 and fcrepo4" https://www.pivotaltracker.com/story/show/5228954513:58
Chris Beer edited "Document CMIS feature" https://www.pivotaltracker.com/story/show/52022147
Chris Beer edited "Update JMeter test to demonstrate starting a transaction, make changes, and commit the changes" https://www.pivotaltracker.com/story/show/4901075713:59
Chris Beer estimated "Tag a 4.0.0-ALPHA1 release of fcrepo4 and publish the artifacts" as 2 points https://www.pivotaltracker.com/story/show/51569881
Chris Beer estimated "Create a hydra-jetty fcrepo4 fork" as 1 point https://www.pivotaltracker.com/story/show/51570127
Chris Beer estimated "Add TX information to the graph response, or introspect the URL for a TX identifier and add the appropriate action..." https://www.pivotaltracker.com/story/show/52316287
Chris Beer estimated "Run profiling against current fcrepo4 master and look for any easy bottlenecks to clear off" as 2 points https://www.pivotaltracker.com/story/show/52325659
Chris Beer estimated "Run benchmarks on SCC cluster at FIZ Karlsruhe with fcrepo3 and fcrepo4" as 3 points https://www.pivotaltracker.com/story/show/52289545
Chris Beer estimated "Stub a migration document that says "coming soon"" as 1 point https://www.pivotaltracker.com/story/show/5157213114:00
Chris Beer estimated "Add import format triples to every node graph (or get that information from the repository resource somehow..)" as..." https://www.pivotaltracker.com/story/show/52316871
<cbeer>eddies1: and i assume we don't need a sprint planning call tomorrow?14:06
<eddies1>nope
<awoods>Here are the results of the AWS cluster tests: https://wiki.duraspace.org/display/FF/Fedora+4+test+results+-+AWS+cluster+of+four++-+2013-06-2514:48
<pivotal-bot_>Andrew Woods finished "Document our AWS benchmarking setup" https://www.pivotaltracker.com/story/show/5156703314:49
Andrew Woods added comment: "Setup documented here:" https://www.pivotaltracker.com/story/show/51567033
Andrew Woods added comment: "Setup documented here: ""14:50
https://wiki.duraspace.org/display/FF/Fedora+4+test+results+-+AWS+cluster+of+four++-..." https://www.pivotaltracker.com/story/show/51567033
Andrew Woods accepted "Document our AWS benchmarking setup" https://www.pivotaltracker.com/story/show/51567033
<eddies>awoods: 4seconds average for a request?!15:08
<awoods>crazy, right!?15:09
<eddies>yah
* fasseg leaves15:50
* github-ff joins16:21
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/eToorw
fcrepo4/master 1cd0ee7 Chris Beer: unit tests for JcrRdfTools
* github-ff leaves
<cbeer>that was ug;y16:23
* travis-ci joins16:28
[travis-ci] futures/fcrepo4#774 (master - 1cd0ee7 : Chris Beer): The build has errored.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/9f0074a060e3...1cd0ee72345f
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8439093
* travis-ci leaves
* github-ff joins16:39
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/M_1WoA
fcrepo4/master beed4ad Chris Beer: add unit test for JcrRdfTools version graph
* github-ff leaves
<pivotal-bot_>Chris Beer finished "Add test coverage for JcrRdfTools" https://www.pivotaltracker.com/story/show/51565867
* travis-ci joins16:44
[travis-ci] futures/fcrepo4#775 (master - beed4ad : Chris Beer): The build has errored.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/1cd0ee72345f...beed4adfd485
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8439704
* travis-ci leaves
<bljenkins>Project fcrepo4 build #896: UNSTABLE in 24 min: http://ci.projectblacklight.org/jenkins/job/fcrepo4/896/16:46
Chris Beer: unit tests for JcrRdfTools
<cbeer>oh powermockito...16:49
* github-ff joins17:12
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/5pIIOg
fcrepo4/master f635e17 Chris Beer: rework JcrRdfTools unit tests to not use powermockito
* github-ff leaves
<cbeer>awoods: i'm going to swap the UUID pid minter for something that mints some hierarchical paths instead17:28
(but still uuid based, i guess..)
so it might mint something like abcd/efgh/12345678/123456789012
<awoods>maybe the option would be nice... as UUIDs are very attractive.17:29
<cbeer>right - i'm not removing the other one
<awoods>can we isolate the existing minter
<cbeer>just making this path-based one the default
* travis-ci joins
[travis-ci] futures/fcrepo4#776 (master - f635e17 : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/beed4adfd485...f635e174cead
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8440981
* travis-ci leaves
<cbeer>(presuming it deals with the performance issue..)
<awoods>ok. Is it clear how a repository manager could configure the repo to use the legacy minter?17:30
<cbeer>as clear as it is to do anything else
(e.g. Spring will magically solve all your problems)
<awoods>If Spring is wiring in the impl, that is great.
<cbeer>i don't think it's a big deal though.. you're already saying you don't care what the identifier is when you ask for fcr:new17:31
and the objects already have their own internal uuid anyway
(of course, it would have helped if i wired it into the right project..)17:32
i guess it's good to set a baseline17:33
<awoods>I am not sure the user is saying "I don't care what the id is". I think the user is saying "I know you are creating a RFC-4122 UUID for me".
<cbeer>they should be greatful it's not just an auto-incrementing id!?!17:34
<awoods>yes they should, ingrates.
<cbeer>of course, for this to make any difference, i guess we'd have to have some overlap in the first segment17:36
back to the drawing board.
<awoods>good point17:37
<cbeer>maybe i just take the first N characters from the UUID
and turn that into a path
so: 87a0a8c317f1e727938fa8-cc85-455f-9877-add54b5db591 => 87a/87a0a8c317f1e727938fa8-cc85-455f-9877-add54b5db591
and we could make the path breaks configurable some day.
<awoods>or pairtree the UUID17:38
<cbeer>hm. could. although then we're making.. 32 objects, worst case.17:39
<awoods>your idea seems like it is on the right path... maybe still some work in coming up with the exact right algorithm.17:42
if we are creating a tree of nodes, how many levels are optimal?17:43
what is the maximum width of any given level before performance breaks down?
<cbeer>width seems to be somewhere around 10k
if i ever got around to dumping my runs into R, i probably could get a real estimate17:44
<awoods>I wonder if the 10k width still holds true for level-2? level-n?
<cbeer>should.17:45
ooh, i know
this is a great dev challenge problem!
<awoods>true
<cbeer>:P
<awoods>we should have dev challenges more often
<cbeer>it's actually a good question to ask the stanford cluster too
just need to make the PID pattern configurable17:46
(via a java property or something)
and i can turn it loose on a bunch of nodes
<awoods>"turn it loose on nodes" = recreate nodes with new paths?
<cbeer>(i really hope someone has done a NOID impl for java)
awoods: sorry, cluster nodes.17:47
compute cluster nodes.
<awoods>sure
<cbeer>aw. there's an unmaintained impl, but it isn't packaged for maven17:48
<awoods>that is all you found?17:49
<cbeer>yeah17:50
and the only java/noid thing on github is a fedora pid minter
but it calls out to the perl webapp
it does seem to make a performance difference, though17:51
just doing a 3-character prefix cut 15s off creating 1000 nodes
<awoods>what is the total time?17:52
<cbeer>1m
so, probably highly variable
<awoods>that is good17:53
<cbeer>but repeatable
i'll try a couple more variations and ship something
we can refine it at OR or before december
<awoods>hit it17:55
<cbeer>guava++
<awoods>what do you like about guava?
<cbeer> final String s = randomUUID().toString();17:57
final Iterable<String> split = Splitter.fixedLength(2).split(s.substring(0, 8));
return Joiner.on("/").join(split) + "/" + s;
better than writing it all myself.17:58
<awoods>so you are also keeping the UUID as the leaf of the path?
<cbeer>yeah. i figured why not.
<awoods>probably a good idea17:59
<cbeer>and supposedly we could parameterize those magic numbers
<awoods>indeed
* github-ff joins18:05
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/OB4doQ
fcrepo4/master dfd898c Chris Beer: Add UUIDPathMinter (and wire it into webapp by default) that prefixes a generated uuid with some branch nodes to help manage level width
* github-ff leaves
* github-ff joins18:09
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/uTdZWQ
fcrepo4/master 1a26123 Chris Beer: make the length and count of the branch node identifiers configurable at the constructor
* github-ff leaves
<cbeer>ok. time to learn about parallel execution18:10
<awoods>does modeshape over threaded configuration?
s/over/offer/
<cbeer>offer what?18:11
<awoods>a way to have it launch multiple threads under the covers.18:12
apparently Ruby does not support threading, correct?
<cbeer>MRI ruby does something like threading, but only one thread is active at a time (although i think IO and other things may happen in the background)18:13
but e.g. jruby has full threading
<awoods>that makes sense
* travis-ci joins18:14
[travis-ci] futures/fcrepo4#777 (master - dfd898c : Chris Beer): The build has errored.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/f635e174cead...dfd898cd7007
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8442656
* travis-ci leaves
* jcoyne leaves18:20
* github-ff joins18:23
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/9UdGsw
fcrepo4/master cf63e95 Chris Beer: use spring (and java properties) to configure the uuid path minter'
* github-ff leaves
* travis-ci joins18:30
[travis-ci] futures/fcrepo4#778 (master - 1a26123 : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/dfd898cd7007...1a26123e0435
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8442827
* travis-ci leaves
* bljenkins leaves18:31
* bljenkins joins18:32
<pivotal-bot_>Chris Beer added "Create UUIDPathMinter that mints UUID identifiers but prefixes them with some artificial hierarchy to increase JCR per..." https://www.pivotaltracker.com/story/show/5235014318:36
Chris Beer estimated "Create UUIDPathMinter that mints UUID identifiers but prefixes them with some artificial hierarchy to increase JCR..." https://www.pivotaltracker.com/story/show/52350143
Chris Beer started "Create UUIDPathMinter that mints UUID identifiers but prefixes them with some artificial hierarchy to increase JCR p..." https://www.pivotaltracker.com/story/show/52350143
Chris Beer finished "Create UUIDPathMinter that mints UUID identifiers but prefixes them with some artificial hierarchy to increase JCR ..." https://www.pivotaltracker.com/story/show/52350143
Chris Beer added comment: "https://github.com/futures/fcrepo4/compare/f635e174cead65bfc481af60b34aa8d10dc43b5f...master" https://www.pivotaltracker.com/story/show/52350143
Chris Beer edited "Create UUIDPathMinter that mints UUID identifiers but prefixes them with some artificial hierarchy to increase JCR pe..." https://www.pivotaltracker.com/story/show/5235014318:37
<cbeer>oh, awoods: where are you backpacking?18:38
<awoods>just out the backyard, in the Eagles Nest Wildness here in Summit County.18:39
3-days
~35miles
Wilderness
cbeer: if you push "Deliver", I will push "Accept"18:42
<cbeer>nice18:45
i'll push deliver after i run some tests with some larger numbers of objects and see how it fares18:46
* travis-ci joins
[travis-ci] futures/fcrepo4#779 (master - cf63e95 : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/1a26123e0435...cf63e95632f2
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8443252
* travis-ci leaves
* ksclarke leaves
<bljenkins>Yippie, build fixed!
Project fcrepo4 build #900: FIXED in 27 min: http://ci.projectblacklight.org/jenkins/job/fcrepo4/900/
Chris Beer: make the length and count of the branch node identifiers configurable at the constructor
<awoods>it is interesting that you would push to master before doing that. Any reason not to test that sort of update locally first?18:47
<bljenkins>Project fcrepo-fixity build #354: STILL FAILING in 51 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/354/
Project fcrepo-fixity-corrupter build #169: STILL FAILING in 1 min 2 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity-corrupter/169/
<cbeer>no reason. i've just been relying on gluck to build the artifacts, because the cluster management environment is not that fast18:49
it's still a clear improvement over the previous config. i'd just like assurances that it makes the large object count better before calling it done-done.18:50
<bljenkins>Project fcrepo-kitchen-sink build #427: STILL FAILING in 3 min 52 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-kitchen-sink/427/18:51
<cbeer>i didn't keep the short leveldb run, but compared to the older configuration, it took 90m to create 10k objects18:58
the new leveldb + uuidpath took 15m
<bljenkins>Project fcrepo-fixity build #355: STILL FAILING in 45 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/355/19:02
Project fcrepo-fixity-corrupter build #170: STILL FAILING in 2 min 26 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity-corrupter/170/19:04
Project fcrepo-kitchen-sink build #428: STILL FAILING in 4 min 21 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-kitchen-sink/428/19:06
<pivotal-bot_>Andrew Woods added "ModeShape performance test: flat hierarchy slowdown" https://www.pivotaltracker.com/story/show/5235209119:15
<cbeer>awoods: https://gist.github.com/cbeer/1400e40420006e2841cc19:40
i'd declare that a pretty good result so far
i suspect a lot of the fluctuation in the new config result is resource contention on the box
(count is the number of objects creates in that second)19:41
<pivotal-bot_>Chris Beer delivered "Create UUIDPathMinter that mints UUID identifiers but prefixes them with some artificial hierarchy to increase JCR..." https://www.pivotaltracker.com/story/show/52350143
Chris Beer added comment: "At 32k objects new / 68k objects old, here is the rate of object creation: https://gist.github.com/cbeer/1400..." https://www.pivotaltracker.com/story/show/5235014319:42
<awoods>cbeer: in the gist, between 15:50 and 16:00 is 10seconds?20:14
<cbeer>no, minutes.
<awoods>got it20:15
2hr/24hr speedup
that is huge20:16
<pivotal-bot_>Chris Beer started "Add TX information to the graph response, or introspect the URL for a TX identifier and add the appropriate actions ..." https://www.pivotaltracker.com/story/show/5231628720:56
Chris Beer finished "Add TX information to the graph response, or introspect the URL for a TX identifier and add the appropriate actions..." https://www.pivotaltracker.com/story/show/52316287
* github-ff joins
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/_6JIVw
fcrepo4/master 4ef7bbb Chris Beer: add tx actions to all node responses
* github-ff leaves
<pivotal-bot_>Chris Beer added comment: "https://github.com/futures/fcrepo4/commit/4ef7bbbb1b6e3674ccbd45a927fa1bb0daace285" https://www.pivotaltracker.com/story/show/52316287
Chris Beer delivered "Add TX information to the graph response, or introspect the URL for a TX identifier and add the appropriate action..." https://www.pivotaltracker.com/story/show/52316287
Chris Beer deleted "Add import format triples to every node graph (or get that information from the repository resource somehow..)" https://www.pivotaltracker.com/story/show/5231687120:58
* ksclarke joins
* awoods leaves21:03
<bljenkins>Project fcrepo-fixity build #356: STILL FAILING in 1 min 53 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/356/21:22
Project fcrepo-fixity-corrupter build #171: STILL FAILING in 3 min 46 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity-corrupter/171/21:24
Project fcrepo-kitchen-sink build #429: STILL FAILING in 7 min 54 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-kitchen-sink/429/21:28
* jcoyne joins21:35
* jcoyne leaves21:36
<cbeer>ok, the 3 hour run finished. 90k objects in 178m21:52
i'll run a 24h run over night.
<bljenkins>Yippie, build fixed!23:53
Project fcrepo4-jmeter build #48: FIXED in 1 hr 6 min: http://ci.projectblacklight.org/jenkins/job/fcrepo4-jmeter/48/
* Edwin Shin: Checkstyle:
* Chris Beer: unit tests for JcrRdfTools
* Chris Beer: add unit test for JcrRdfTools version graph
* Chris Beer: rework JcrRdfTools unit tests to not use powermockito
* Chris Beer: Add UUIDPathMinter (and wire it into webapp by default) that prefixes a generated uuid with some branch nodes to help manage level width
* Chris Beer: make the length and count of the branch node identifiers configurable at the constructor
* Chris Beer: use spring (and java properties) to configure the uuid path minter'
* Chris Beer: add tx actions to all node responses

Generated by Sualtam