Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
<bljenkins>Project fcrepo-fixity build #370: FAILURE in 8 hr 51 min: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity/370/01:15
Project fcrepo-kitchen-sink build #442: FAILURE in 14 hr: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/442/
Project fcrepo4-jmeter build #50: FAILURE in 2 hr 57 min: http://ci.fcrepo.org/jenkins/job/fcrepo4-jmeter/50/01:45
* Edwin Shin: I'm not sure why these servlet classes were inlined, but off they go
* Chris Beer: JunitTest for versioning
* Chris Beer: Removed setters that were only used in tests, added Testhelper mathod to set fields via reflection and changed the tests to use the new method
* Chris Beer: Remove unnecessary FedoraRepositoryImportTest and just rely on the FedoraImportTest instead
* Chris Beer: bump to ISPN 5.3.0.Final
Yippie, build fixed!01:53
Project fcrepo-fixity build #371: FIXED in 8 min 42 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity/371/
Yippie, build fixed!01:54
Project fcrepo-kitchen-sink build #443: FIXED in 8 min 54 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/443/
* ksclarke leaves01:58
* leenata joins02:39
* fasseg joins02:46
eddies: so some numbers came up in Vienna yesterday about Asger's use cases: one project has 400h of video per day, for which they have a large backlog, and in a future project they will have 13 billion images per year from newspapers...02:55
so he is very keen on fedora 4 and the possibility to scale out.02:56
and has anyone ever tried Apache Synapse for load balancing?03:00
<eddies>never heard of it03:20
<fasseg>eddies: arg please ignore the mail I just send you CC, no idea how you got in cc there03:46
<pivotal-bot_>Frank Asseg started "Run profiling against current fcrepo4 master and look for any easy bottlenecks to clear off" https://www.pivotaltracker.com/story/show/5232565903:54
<fasseg>hmm it seems jersey needs 16% of the overall time which in my mind is too much: https://docs.google.com/file/d/0B5nd_qlYdcqyTW5oMG9lVk9xZmc/edit?usp=sharing04:14
<pivotal-bot_>Frank Asseg added comment: "Jersey seems to need 16% of overall time, which seems too much: https://docs.google.com/file/d/0B5nd_qlYdcqy..." https://www.pivotaltracker.com/story/show/52325659
<fasseg>This class seems to take 10% of the overall time which is a bit hefty: com.codahale.metrics.jersey.InstrumentedResourceMethodDispatchProvider$TimedRequestDispatcher.dispatch(Object, HttpContext)04:21
<eddies>fasseg: hrm. that's unfortunate04:27
<fasseg>can I just remove the config bean for codahale metrics. <bean class="org.fcrepo.metrics.MetricsConfig"/> and it will not get used?04:28
<eddies>our metrics are slowing down our metrics?
<fasseg>sigfnificantly
<eddies>not sure. i guess that should work
<fasseg>ill give it a shot04:29
<eddies>we don't *have* to use metrics-jersey to instrument either
that was just a convenience for jersey integration. i wonder if the other, non-jersey specific instrumentation is lighter-weight04:30
if we move to jersey-2.0 we'll probably have to do that anyway
<fasseg>right
Ill investigate and maybe create a ticket later if it's not an easy fix?04:32
I ask you cause awoods isnt here ;)
are you doing this on your free time atm btw?
oh did building fcrepo4 without tests get faster? Nice!04:34
hmm removing the config bean didn't help :/04:36
oh and now I just saw that I added the SCAPE meeting in Vienna to my own Calendar, and not the fedora-futures calendar.....05:07
very sorry about that!
<pivotal-bot_>Frank Asseg added comment: "Part of the time spent in jersey is generating metrics using codahale's library for jersey. Just removing th..." https://www.pivotaltracker.com/story/show/5232565905:13
<fasseg>so yeah removing @Timed decreased response time in a quick run by about 8%
<eddies>fasseg: sorry, was afk05:17
<fasseg>no worries i just keep rambling...
<eddies>today is technically my last paid day =)
so you measured ~8% difference. what does that actually amount to in real time?05:18
<fasseg>I have to rerun jmeter tests with @timed annotation then I can give you some hard numbers but for me atm it should be around 5ms of ~7705:19
for simple creates I have already thrown away the numbers from the standard test but will rerun it05:20
hmm nice things happened I guess, I can't see anymore of the increase in request time after 6000 objects now, avg, throughput and median are stable!05:23
<eddies>chris updated the uuidpidminter so that objects are distributed hierarchically
see: https://gist.github.com/cbeer/113f01f1de0c73ba64ff05:25
<fasseg>yeah just looking at it....05:29
nah not that one though05:30
was looking at that one: https://github.com/futures/fcrepo4/commit/dfd898cd7007cb8e31c4be2d5aa70c4f62555b00
OI but I ran into an exception after ~7k objects05:32
https://gist.github.com/fasseg/588358205:34
* github-ff joins05:35
[fcrepo-bagit-object-serialization] eddies pushed 1 new commit to master: http://git.io/GAZsDg
fcrepo-bagit-object-serialization/master e25d446 Edwin Shin: Removes logback dependency from compile scope (we shouldn't be including a concrete slf4j impl here)
* github-ff leaves
<eddies>that's weird. weirder still that i was just removing logback from fcrepo-bagit-object-serialization05:37
but that couldn't have affected you =)
<fasseg>nah I didnt even pull you rchange yet05:38
<eddies>is the logback jar in your WEB-INF/lib?
<fasseg>classic and core jars are there yes, and I do see log output05:39
maybe oom?
<eddies>no idea05:40
<bljenkins>Project fcrepo-bagit-object-serialization build #20: SUCCESS in 5 min 46 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-bagit-object-serialization/20/05:41
<fasseg>yeah I guess it's a problem with the JVM I cant even connect via Jconsole after the exception occured05:43
<bljenkins>Project fcrepo-kitchen-sink build #444: SUCCESS in 6 min 21 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/444/05:47
* github-ff joins05:49
[fcrepo4] eddies pushed 1 new commit to master: http://git.io/GalNCQ
fcrepo4/master f0235b5 Edwin Shin: Only our deployable (i.e. webapp) should pull in a concrete slf4j runtime impl
* github-ff leaves
* kaarefc joins05:50
<bljenkins>Project fcrepo-sequencer-archives build #15: SUCCESS in 4 min 37 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-sequencer-archives/15/05:52
<fasseg>nah only a third of heap and a quarter of non heap space is taken05:54
so no memory issues
but I can reproduce this on my system, and it happened after ~6400 object creations 3 times now05:55
But a read error seems to occor since: t org.infinispan.util.logging.Log_$logger.errorReadingFromFile(Log_$logger.java:1504)05:57
<pivotal-bot_>Edwin Shin added "Remove multiple slf4j bindings" https://www.pivotaltracker.com/story/show/5252718906:00
Edwin Shin started "Remove multiple slf4j bindings" https://www.pivotaltracker.com/story/show/5252718906:01
Edwin Shin accepted "Remove multiple slf4j bindings" https://www.pivotaltracker.com/story/show/52527189
* travis-ci joins06:02
[travis-ci] futures/fcrepo4#806 (master - f0235b5 : Edwin Shin): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/f92e90729158...f0235b530a75
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8534687
* travis-ci leaves
<eddies>fasseg: i don't think i've ever heard of intermittent classdefnotfound errors06:03
<fasseg>He got the same probs and it was related to the versions it seems: java.lang.NoClassDefFoundError: ch/qos/logback/classic/spi/ThrowableProxy06:04
arg sry link here: http://lists.geomajas.org/pipermail/majas-dev/2012-March/004788.html
<eddies>well, i suppose if you were mucking w/ classloaders
ah that's interesting
<fasseg>but i can only find one version of the jar in my tomcat classpath lib and WEB-INF/lib06:05
<eddies>yeah, i just checked my fcrepo-webapp. I only see logback-classic:1.7.5, logback-core:1.7.5, slf4j-api:1.7.5 and jcl-over-slf4j:1.7.506:06
and that looks right to me06:07
are you running in tomcat?
or via mvn jetty:run?
<fasseg>tomcat06:08
I did this: [ruckus@agrajag ff-jmeter-madness (master *)] find /home/ruckus/dev/tomcat7/ -iname *logback*
<eddies>any other webapps running?
<fasseg>/home/ruckus/dev/tomcat7/webapps/fcrepo/WEB-INF/classes/logback.xml
/home/ruckus/dev/tomcat7/webapps/fcrepo/WEB-INF/lib/logback-core-0.9.29.jar
/home/ruckus/dev/tomcat7/webapps/fcrepo/WEB-INF/lib/logback-classic-0.9.29.jar
so only these two jars
nah
and only two jars for slf4j06:09
<eddies>huh. b/c the resolution reported in that thread was another webapp in tomcat that was pulling in different logback
<fasseg>yeah but I got nothing there, let me try with jetty06:11
find /home/ruckus/dev/tomcat7/ -iname *slf4j* -or -iname *logback* -> only the four jars in fcrepo-webapp06:12
<eddies>fasseg: can you also reply to adam on https://github.com/futures/fcrepo4/pull/86 ?06:16
he says he's waiting for you to confirm before proceeding w/ review
<fasseg>I dont get it....confirm what? there's no question on the pull request. but I guess I just add that yeah it's finished...hence the pull request06:19
<bljenkins>Project fcrepo-fixity-corrupter build #187: SUCCESS in 2 min 31 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/187/
Project fcrepo-kitchen-sink build #445: SUCCESS in 8 min 10 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/445/06:25
Project fcrepo-fixity build #372: SUCCESS in 9 min 8 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity/372/06:26
<fasseg>so it didnt happen in the default configuration but using the fast-ingest configuration, Im just retrying with jetty and the fast-configuration to make sure that it's not related to tomcat but the config06:35
tomcat7 with APR lib seems to be a bit faster fo me than jetty btw06:37
ok the problem happens with jetty also and it's still java.io.FileNotFoundException: target/FedoraRepository/storage/FedoraRepository/-493350912 (Too many open files), I guess 16k open files is not enough ill check the sources for the hardcoded limit chris mentioned06:47
but only using the fast-configuration for me06:48
and I guess once the hard coded limit is reached performance will decrease...
Okay I see: https://issues.jboss.org/browse/ISPN-1300: This reduces the number of files to 2^22 = 4,194,304 files now thats a limit06:53
ah no thats not the number of open files...
phew
I guess ill just use the other backend for the fast config as well ;)06:55
oh this is blazingly fast!06:59
avg 10ms for object creation test ill run the full test in a minute07:00
created 10k object in 80 secs.....07:02
<eddies>fasseg: you mean the leveldb cache store?07:03
<fasseg>aye, with query disabled
<eddies>yeah, go ahead and update the fast config to use that =)07:04
afk
<fasseg>kk
and ill rename it to single-no-indexing as well since now that's the only difference07:06
oh this is sooo nice: full test with a loopcount of 200 gives me a 28 avg and a 31 median07:13
* github-ff joins07:19
[fcrepo4] fasseg pushed 1 new commit to master: http://git.io/RU1uJQ
fcrepo4/master 1ca3fa9 fasseg: changed fast-ingest configuration to use leveldb cachestore
* github-ff leaves
* unai_ joins07:29
<fasseg>Should the asnc-indexing configuration use the leveldb cahcestore as well?
<unai_>First of all, hello to everyone. Here Unai, trying to develope something to connect SCAPE with Fedora07:31
I would try to be short. I am trying to create some nodes and version in fcrepo4, but I don't really understand how to do this.07:32
In the wiki says (https://wiki.duraspace.org/display/FF/REST+API#RESTAPI-Versioning) smth, but is not really clear.
* travis-ci joins
[travis-ci] futures/fcrepo4#807 (master - 1ca3fa9 : fasseg): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/f0235b530a75...1ca3fa994ab6
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8536469
* travis-ci leaves
<bljenkins>Project fcrepo-fixity-corrupter build #188: SUCCESS in 56 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/188/07:33
<unai_>I create an object like this : curl -X POST http://localhost:8080/fcrepo/rest/objects/q1
After that I tag this version: curl -X POST http://localhost:8080/fcrepo/rest/objects/q1/fcr:versions/v107:34
Change something on it: curl -X POST -H "Content-Type:application/sparql-update" http://localhost:8080/fcrepo/rest/objects/q1 -d "INSERT {<http://localhost:8080/fcrepo/rest/objects/q1> <scape:bertsiokprobatzen> \"proba1\" } WHERE {}"
Add a new tag this version: curl -X POST http://localhost:8080/fcrepo/rest/objects/q1/fcr:versions/v207:35
And take a look to versions list: curl -X GET http://localhost:8080/fcrepo/rest/objects/q1/fcr:versions
<bljenkins>Project fcrepo-fixity build #373: SUCCESS in 4 min 9 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity/373/07:36
<unai_>And I get somth like that:07:37
<http://localhost:8080/fcrepo/rest/jcr:system/jcr:versionStorage/8c/cf/50/8ccf502a31969893eb6d8fd0bc850b0bccfa4ee6/jcr:rootVersion/jcr:frozenNode> fedora-internal:frozenMixinTypes "fedora:resource" , "fedora:object" ;...
fedora-internal:hasVersionLabel "v1" , "v2" ;07:38
.....
fedora-internal:uuid "87a0a8c317f1e7e94c8d91-d3d8-4df8-8ba3-901bb0697206" . <http://localhost:8080/fcrepo/rest/objects/q1> fedora-internal:hasVersion <http://localhost:8080/fcrepo/rest/jcr:system/jcr:versionStorage/8c/cf/50/8ccf502a31969893eb6d8fd0bc850b0bccfa4ee6/jcr:rootVersion/jcr:frozenNode> .
before continuing writting maybe unuseful things, I would appreciate some ideas about how can I create versions07:40
of files, and how to get them also via rest api
Thanks in advance to everyone
<fasseg>Ill relay your question when people arrive, unai!07:41
<eddies>unai_: we haven't really worked on versions yet. we get some of the basics just by sitting on top of jcr, but there's very little that's fedora-specific that's been implemented yet.
and that's in part because folks, including FIZ, have talked about wanting new ways for versioning to work in fedora 4 vs fedora 3, but that has yet to manifest as concrete, implementation-focused requirements07:43
<fasseg>yeah we requested top level versioning for fedora4
and looking at the FedoraVersionsIT it seems the capability exists....since we could just tag the parent object with a version, as I understood it....07:45
or wont this apply to the whole ubgraph yet and still needs to be done?
*subgraph
<eddies>i dunno. i really haven't worked w/ it07:46
i'd expect the basics at the node level to work though
if it doesn't, it's a bug
although i though leenata just put together some tests07:49
<fasseg>unit tests...
<eddies>unai_: just looking at what you posted, what's the problem, exactly?07:52
a GET on /<resource>/fcr:versions returns you the list of versions (v1, v2) right?07:53
<unai_>The problem is somehow that I am not able to see the versions with jena.
<eddies>and does a GET on /<resource>/v1 and /<resource>/v2 not return what you expect?
<unai_>Yes, thats rigth
I will check07:54
<eddies>sorry, should have been GET on /<resource>/fcr:versions/v1
and /<resource>/fcr:versions/v2
<unai_>It works. Returns something similar to the text that I write on the top07:57
<eddies>unai_: so…all good for now?08:00
<unai_>Ok, I will check some things and I will be back, before to start doing basic questions :)
* github-ff joins08:01
[fcrepo4] leenata created junit-fedoraresource-extention (+4 new commits): http://git.io/cPD4Rg
fcrepo4/junit-fedoraresource-extention 79c7905 Leenata Bankhele: JunitTest for versioning
fcrepo4/junit-fedoraresource-extention 9c71a6d fasseg: Mocked call to Variant.getMediaType()
fcrepo4/junit-fedoraresource-extention 8bc8ce8 Leenata Bankhele: Junit tests for versioning
* github-ff leaves
<unai_>Thanks edie
<eddies>unai_: incidentally, all of what's implemented is essentially here: https://github.com/futures/fcrepo4/blob/master/fcrepo-http-api/src/main/java/org/fcrepo/api/FedoraVersions.java
feel free to open new tickets in tracker if you like (or even better, submit pull requests ;-__08:02
<unai_>ok, I will check that
<eddies>err, that was supposed to be ;-)
* unai_ leaves08:09
<eddies>leenata: are you using eclipse?08:15
<bljenkins>Project fcrepo-fixity-corrupter build #189: SUCCESS in 1 min 12 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/189/08:16
<leenata>hey08:25
yes
<fasseg>hmm defaultFilter also generates a lot of PathNotFoundExceptions for me in DefaultFilter.apply():6408:51
7k for 720 create Object/Datastream requests08:52
so were doing flow control by exception handling...:/08:53
<eddies>afk: going out for dinner. bbl08:57
<fasseg>enjoy!
* kaarefc leaves09:09
* github-ff joins09:24
[fcrepo4] fasseg force-pushed perf-defaultfilter from 00a7aac to eab678c: http://git.io/mTjTCA
fcrepo4/perf-defaultfilter eab678c fasseg: added check for the event path to decrease the amount of Session.getNode() calls in the DefaultFilter
* github-ff leaves
* ksclarke joins09:48
<bljenkins>Project fcrepo-fixity-corrupter build #190: SUCCESS in 1 min 15 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/190/
<fasseg>adding reader-strategy : "non-shared" for single gives a small perf boost as well it seems09:57
but onyl about 2% for me09:58
the strange thing is that this should be according to the documentation "not-shared" can be expensive10:04
* github-ff joins10:25
[fcrepo4] fasseg created tweak-indexing (+1 new commit): http://git.io/u-1y2g
fcrepo4/tweak-indexing b11d2a2 fasseg: changed single configuration for slight perfomance increase
* github-ff leaves
* leenata leaves10:26
* github-ff joins
[fcrepo4] fasseg opened pull request #89: Changed single configuration for slight perfomance increase (master...tweak-indexing) http://git.io/aT7rmA
* github-ff leaves
<bljenkins>Project fcrepo-fixity-corrupter build #191: SUCCESS in 1 min 25 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/191/10:47
<cbeer>eddies: no, versioning responses don't work as expected.. we never implemented the checkin/checkout stuff11:35
(partly because we don't know anything about JCR versioning)11:37
eddies: and i'm running the urandom thing now12:13
eddies: i'd say urandom has little to no impact on the timings in the cluster.12:51
which is not to say it wouldn't have an impact elsewhere12:52
<eddies>it may not have enough relative impact and also on anything long-running/large enough like the cluster, there may be enough entropy anyway13:39
* fasseg leaves14:03
<pivotal-bot_>A. "Torbulater" Soroka added comment: "@frankasseg : Looks like the build is failing?15:51
https://travis-ci.org/futures/fcrepo4/builds/8539714" https://www.pivotaltracker.com/story/show/51983823
Chris Beer started "Add Transactions support to Rubydora" https://www.pivotaltracker.com/story/show/4901078116:03
Chris Beer deleted "Test <UDP> and <MPING> on the futures cluster." https://www.pivotaltracker.com/story/show/4773191516:04
Chris Beer finished "Add Transactions support to Rubydora" https://www.pivotaltracker.com/story/show/4901078116:47
Chris Beer added comment: "https://github.com/projecthydra/rubydora/commit/7fc66ac1ba299219ebfa1e66a44e8b1a221c0b02" https://www.pivotaltracker.com/story/show/4901078116:48
* github-ff joins17:50
[fcrepo4] cbeer pushed 2 new commits to master: http://git.io/rjAWzQ
fcrepo4/master 904ad33 Chris Beer: make sure Session#impersonate within a transaction returns the new session within the transaction too
fcrepo4/master 21a947d Chris Beer: logging for fcr:tx endpoint
* github-ff leaves
<pivotal-bot_>Chris Beer delivered "Add Transactions support to Rubydora" https://www.pivotaltracker.com/story/show/4901078117:51
* travis-ci joins18:06
[travis-ci] futures/fcrepo4#810 (master - 21a947d : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/1ca3fa994ab6...21a947d2d19c
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8555838
* travis-ci leaves
<bljenkins>Project fcrepo4 build #928: UNSTABLE in 16 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/928/
* Chris Beer: make sure Session#impersonate within a transaction returns the new session within the transaction too
* Chris Beer: logging for fcr:tx endpoint
* github-ff joins18:33
[fcrepo4] cbeer pushed 4 new commits to master: http://git.io/Rw9KWQ
fcrepo4/master a6d1313 Chris Beer: remove testing-only session factory methods
fcrepo4/master c7a4c87 Chris Beer: remove test-only methods
fcrepo4/master b7712e1 Chris Beer: add javadocs to the FedoraObjectSerializer
* github-ff leaves
* travis-ci joins18:45
[travis-ci] futures/fcrepo4#811 (master - 2f6b89f : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/21a947d2d19c...2f6b89fb8e40
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/8557005
* travis-ci leaves
<bljenkins>Yippie, build fixed!18:47
Project fcrepo4 build #929: FIXED in 14 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/929/
* Chris Beer: remove testing-only session factory methods
* Chris Beer: remove test-only methods
* Chris Beer: add javadocs to the FedoraObjectSerializer
* Chris Beer: javadoc'ing
Project fcrepo-fixity-corrupter build #192: SUCCESS in 2 min 1 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/192/18:49
* ksclarke leaves19:37
* ksclarke joins21:04