Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* bljenkins leaves02:12
* bljenkins joins02:23
* kaarefc joins02:42
* eddies leaves04:33
* eddies joins04:41
* eddies leaves
* eddies joins
* fasseg joins06:18
* eddies leaves06:42
<pivotal-bot>Frank Asseg added comment: "Actually while investigating possibilities, I stumbled upon https://code.google.com/p/maven-java-formatter-p..." https://www.pivotaltracker.com/story/show/5172054507:00
* eddies joins
* eddies leaves
* eddies joins
<fasseg>eddies: for code formatting there's also the maven plugin https://code.google.com/p/maven-java-formatter-plugin/ which can reformat code on build based on a eclipse configuration, which we have available already...07:01
<eddies>hey that's kind of cool. hadn't heard of that before07:02
<fasseg>me neither, and it works nice on fcrepo-fixity...with just a small plugin config and the addition of the fedora-formatter.xml file07:03
<eddies>fasseg: any progress on the performance front?07:05
<fasseg>and people could use their own eclipse formatter configuration.....as long as they run the plugin before actually committing
just running some tests in the bg
but havent hit the golden well yet
everything i try except "query: disable" doesn't boost perf as hoped
and im gettting quite sure that the massive I/O is solely due to lucene :/07:07
<eddies>only possible issue w/ the formatter plugin is that the release notes mention eclipse 3.5 compatibility (from back in 2011). so i wonder if there are any issues applying rules between this plugin and a more recent version of eclipse (Juno is v4.2)07:08
<fasseg>and async isnt really an option either, since at the rate lucene is producing load on my system, the repo will be under heavy I/O until lucene has endexed everything
<eddies>well, that may be interesting to note for a clustered config
<fasseg>yeah an 0.4 is in dev....07:09
which was for ecl 3.8
<eddies>(as in, i wonder if you can offload async indexing to another code)07:12
<fasseg>or jst feed a SOLr with that info, and dont index in mode?07:13
because at least it seems to me that a lot of stuff is indexed which will never be exposed for a search07:14
like internal jcr properties
primaryTypes and that kind of stuff07:15
but I'd have to take a look in the indexing code in mode for investigating that
because in my mind lucene should not increase the request times on my local box from 60(!!) ms to 400ms, there has to be some uneccessary work in there .)07:17
and this even on blank nodes
eddies: I enabled <async enabled="true" flushLockTimeout="15000" threadPoolSize="5"/> in infinispan.xml and now I get about 30ms Response Times
but this is without queries and without the defaultfilter atm, lemme try again with these...07:57
yeah and ofc bad perf will catch up with you in the long run....since when the threadpool is exhausted you start blocking :/08:48
* awoods joins08:56
<fasseg>cbeer: Is there a rationale for using <property name="fsyncMode" value="perWrite" /> ? Since <property name="fsyncMode" value="default" /> yields better performance for me...09:15
in the infinispan.xml configuration btw :)09:21
* kaarefc leaves09:29
* kaarefc joins09:49
<cbeer>fasseg: at the expense of not having things necessarily written to disk before the request is over.10:16
so, where do we want to be on the speed vs durability issue?
<fasseg>but that's the async -bit right?
I stashed that already, since "ofc bad perf will catch up"10:17
or is that about fSyncMode ?
so with fSyncMode="default" and a crash before the OS buffer is full or the object is read would result in inconsistent state10:19
* kaarefc leaves
* kaarefc joins
<cbeer>async is about when things get written to disk
fsync is what happens when they are written
<fasseg>right.... and async perf will evt be the same on large scale ingest since when the threadpool for the async worker is exhausted it just blocks a single request for a long time10:20
<cbeer>makes sense.10:21
<fasseg>but fsync=default sounds safe to me, since you wouldn't create inconsistent state unless there was a crash, and you would only lose a single OS buffer
so it would only impact one single write to a datastream or sth
but this depends on the OS too i guess10:22
and ofc the size of the buffer
<cbeer>(there was also something about ISPN keeping too many file handles open if you don't fsync-per-write.. but they may have fixed that)
fasseg: re: "because in my mind lucene should not increase the request times on my local box from 60(!!) ms to 400ms, there has to be some uneccessary work in there"10:23
have you tried raising the eviction level, maybe?
to keep more objects around in RAM
<fasseg>ill try
* kaarefc leaves10:24
<fasseg>raising maxEntries fom 500 to 50000 didnt't help10:28
<pivotal-bot>Chris Beer edited "Within a transaction, graph subjects should include the transaction identifier for all node-subjects." https://www.pivotaltracker.com/story/show/5171751310:41
Chris Beer added comment: "stashed in https://github.com/futures/fcrepo4/commit/43f83850b5041bd48f30d9a758cd0cbaec458433" https://www.pivotaltracker.com/story/show/5167523510:42
Chris Beer finished "Wire in a JAX-RS resource to manipulate JCR workspaces" https://www.pivotaltracker.com/story/show/51675235
Chris Beer delivered "Add unit test coverage to uri resource graph injection classes" https://www.pivotaltracker.com/story/show/51566849
Chris Beer added "java.lang.NullPointerException at org.fcrepo.utils.impl.CacheStoreEntry.getExternalIdentifier(CacheStoreEntry.java:87)" https://www.pivotaltracker.com/story/show/5181245710:44
Chris Beer started "java.lang.NullPointerException at org.fcrepo.utils.impl.CacheStoreEntry.getExternalIdentifier(CacheStoreEntry.java:87)" https://www.pivotaltracker.com/story/show/51812457
* gregjansen joins10:48
<pivotal-bot>Chris Beer edited "fcrepo4 stress test" https://www.pivotaltracker.com/story/show/5068254710:53
Chris Beer added comment: "Ran fcrepo4 for 24 hours in various configurations. Will report to the list after the conclusion of the curre..." https://www.pivotaltracker.com/story/show/5068254710:54
<barmintor>what? what have I not done now?
<cbeer>barmintor: still debugging the ISPN distributed execution stuff11:13
<barmintor>oh, that.
I blame the requester.
<cbeer>yeah, he didn't do a good job reviewing the request11:14
<barmintor>cbeer: is the issue identifying the binary store?11:15
<cbeer>barmintor: yes: https://github.com/futures/fcrepo4/blob/master/fcrepo-kernel/src/main/java/org/fcrepo/services/functions/CacheLocalTransform.java#L53 is returning null in a chained configuration11:17
and Cache<K, V> cache is empty11:18
<barmintor>cbeer: well, hell. I'll have to rummage around in MODE to fix that. I might be able to do it this afternoon, or Wednesday.11:20
<cbeer>barmintor: ok. i'm going to spend the rest of standup trying to figure out what's going on, and i'll push the broken test11:21
* gregjansen leaves11:22
<pivotal-bot>Edwin Shin accepted "Use component scanning to inject required services" https://www.pivotaltracker.com/story/show/5156814311:26
Edwin Shin accepted "Stop recreating a test namespace every time a JAX-RS resource is created." https://www.pivotaltracker.com/story/show/51571173
Edwin Shin accepted "Inject Session in FedoraTransactions" https://www.pivotaltracker.com/story/show/51646461
Edwin Shin accepted "Create canonical jmeter test" https://www.pivotaltracker.com/story/show/51736203
<fasseg>eddies: http://docs.jboss.org/infinispan/5.1/apidocs/org/infinispan/loaders/file/FileCacheStoreConfig.html11:29
* nbanks joins11:33
<fasseg>there's also this page which I looked at: https://community.jboss.org/wiki/HowToTuneModeShapeForBetterPerformance11:41
<cbeer>that's for mode 2.x though, right?11:58
<pivotal-bot>Edwin Shin added comment: "Reconfigured the historical dashboard to highlight performance metrics: http://sonar.fcrepo.org/dashboard/con..." https://www.pivotaltracker.com/story/show/5151775312:01
<cbeer>barmintor: so, it looks like when running in DEF, you don't get access to the cache loader manager12:16
so no cache configuration info...
* github-ff joins12:23
[fcrepo4] cbeer created npe-for-barmintor-to-look-at (+1 new commit): http://git.io/f_xHuA
fcrepo4/npe-for-barmintor-to-look-at 07174ce Chris Beer: REMOVE THIS COMMIT - demonstrate NPE when running CacheLocalTransform on a distributed, chained cache
* github-ff leaves
<cbeer>barmintor: i see the NPE running FedoraResourceIT.tetDatastreamGraph
* github-ff joins12:24
[fcrepo4] cbeer deleted tx-service at 2aed1ac: http://git.io/rfKe4w
* github-ff leaves
* github-ff joins
[fcrepo4] cbeer deleted distexec at 83b1ff5: http://git.io/_vASLw
* github-ff leaves
* github-ff joins
[fcrepo4] cbeer deleted HTMLAllTheRDF at 1620818: http://git.io/IcUkUQ
* github-ff leaves
* github-ff joins
[fcrepo4] cbeer deleted rdf-describe at 4eb23f0: http://git.io/ea5mNQ
* github-ff leaves
* github-ff joins12:25
[fcrepo4] cbeer deleted rdf-all-the-things at d0b7c26: http://git.io/-Tn_zw
* github-ff leaves
* github-ff joins
[fcrepo4] cbeer deleted ProviderForSession at 0c8cf26: http://git.io/V4bc_A
* github-ff leaves
* github-ff joins
[fcrepo4] cbeer deleted 49012389-workspace at 986cc0f: http://git.io/_adO_A
* github-ff leaves
* github-ff joins12:26
[fcrepo4] cbeer deleted 49205649 at f6bef32: http://git.io/m9T0NQ
* github-ff leaves
* github-ff joins
[fcrepo4] cbeer deleted html-template-annotation at 231da5b: http://git.io/UindWQ
* github-ff leaves
<eddies>somehow, i missed this: https://docs.jboss.org/author/display/MODE/Monitoring12:39
we should be able to put together an impl of https://github.com/ModeShape/modeshape/blob/master/modeshape-jcr-api/src/main/java/org/modeshape/jcr/api/monitor/RepositoryMonitor.java and get some metrics direct from mode12:46
<awoods>eddies: that is a great find
<eddies>i kinda wish they were using the metrics library rather than having cooked up their own. then the reporting out would be more "standard"12:47
"patches welcome", i suppose12:48
<bljenkins>Project fcrepo-fixity-corrupter build #116: SUCCESS in 50 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity-corrupter/116/
Project fcrepo-fixity build #300: SUCCESS in 4 min 54 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/300/12:52
<cbeer>eddies: nice12:53
ok, afk for a couple hours12:55
barmintor: if nothing obvious comes to you about that NPE, i'm happy to just patch it over and call it good
<bljenkins>Project fcrepo-fixity-corrupter build #117: SUCCESS in 41 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity-corrupter/117/13:03
Project fcrepo-fixity build #301: SUCCESS in 5 min 27 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/301/13:08
Project fcrepo-kitchen-sink build #396: SUCCESS in 6 min 28 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-kitchen-sink/396/13:10
* nbanks_ joins13:24
* nbanks leaves13:26
* nbanks_ leaves13:54
* nbanks joins
* nbanks_ joins14:00
* nbanks leaves
* nbanks_ leaves14:11
* nbanks joins14:12
* nbanks_ joins14:32
* nbanks leaves14:34
* fasseg leaves15:45
* bljenkins leaves15:51
* bljenkins joins15:52
* nbanks joins16:08
* nbanks_ leaves16:10
* nbanks leaves16:45
* nbanks joins17:11
* nbanks leaves17:17
<cbeer>ok, back.17:32
awoods, eddies: i'm confused by the transactions thread. should i try to understand it, or did you guys sort it out in the end?17:35
<awoods>cbeer: good question... I do not think there was resolution outside of the fact that the question should be addressed/resolved.17:36
There seems to be momentum in the direction of retaining the tx-id URL prefix17:37
<cbeer>ok. so we're back to my question about how the graph subjects should look
<awoods>There is a "developer-centric" concern that client code should not have to be transaction-aware.
cbeer: yes
<cbeer>i wonder.. maybe within a transaction, we could just add a same-as triple to the non-tx resource17:38
and call it good
<awoods>that may do the trick17:39
Certainly within the scope of a transaction, it probably makes sense to have the tx-id included in the graph subject URI
<cbeer>and then we'd have the option of honoring non-tx-prefixed subjects or not
<awoods>you mean, using the same-as triple would give us the option of honoring non-tx-prefixed subjects, or not?17:40
<cbeer>yes, in sparql-update requests or something17:41
but we can also pretend we're not obligated to
and not implement that feature yet.
<awoods>my concern was around how we handle requests on resources that have the tx-id prefix, but the transaction has been closed.17:42
<cbeer>awoods: i don17:43
hm. i was about to say i don't think we need to care
but if we're pretending you can hand these resources off to someone else
<cbeer>who is unaware of the transaction
i guess the client may have an obligation to keep a tx alive until services it handed stuff to are complete17:44
although we could keep a list of dead tx for some configurable length of time, i suppose
* nbanks joins
<awoods>or we could catch the 404 and see if there is a tx-id prefix in the request...17:45
<cbeer>ah, and not try to validate it at all?
that'd work for me17:46
<awoods>just brainstorming
<cbeer>i think it arguably makes sense... with an opaque tx id, how is a client ever to know if the transactioned ever existed or not17:47
so there's no reason we shouldn't lie to them.
<awoods>that is one way of putting it
in any case, the handling of 404s on tx-id resources is its own ticket...17:48
<awoods>but I have not heard much argument for handling the communication of tx-ids in any other way than via URI prefixes.
* nbanks leaves
<cbeer>and as long as we're fine with different graph subjects within a tx, i think that's fine17:50
<awoods>what is the example of having different graph subjects within a tx?
<cbeer>see my initial email17:51
<awoods>you mean, with or without the tx-id prefix?
<cbeer>and i think a 410 Gone is probably more appropriate than a 3xx17:52
because it really /isn't/ the same resource.
<awoods>I would agree, except it would be helpful to give a hint to a client that is requesting /tx-id/abc that maybe the resource can be found at /abc.17:54
<cbeer>but if i asked service X to do this in the scope of a transaction, and the transaction has gone away, i'm not sure i'd want that client trying to recover on its own17:56
but in the case where the resource URI was stored by the client, or passed to another service that wanted to retrieve that resource much later in the future, it may be help to provide a hint as to where the actual resource may be found.17:57
<cbeer>303 See Other?18:05
This response indicates that the correct response can be found under a different URI and should be retrieved using a GET method. The specified URI is not a substitute reference for the original resource.18:06
<barmintor>(i do that sometimes)18:32
GAAAAH I quit. See you tomorrow.18:37
* barmintor leaves
* bljenkins leaves18:43
* bljenkins joins18:44
<cbeer>mock mock mock18:52
stub stub stub
* nbanks joins19:16
* nbanks leaves19:21
<pivotal-bot>Chris Beer edited "java.lang.NullPointerException at org.fcrepo.utils.impl.CacheStoreEntry.getExternalIdentifier(CacheStoreEntry.java:87)" https://www.pivotaltracker.com/story/show/5181245719:26
* github-ff joins19:39
[fcrepo4] cbeer force-pushed tx-workspace-aware-http-subjects from 43f8385 to 706d838: http://git.io/xa-QhQ
fcrepo4/tx-workspace-aware-http-subjects 706d838 Chris Beer: Include transactions and workspace identifiers in HTTP Graph Subjects....
* github-ff leaves
* bljenkins leaves19:41
* bljenkins joins19:42
<pivotal-bot>Chris Beer added comment: "Included in https://github.com/futures/fcrepo4/pull/83/files#L10R25" https://www.pivotaltracker.com/story/show/5167523519:44
Chris Beer added comment: "https://github.com/futures/fcrepo4/pull/83/files#L10R25" https://www.pivotaltracker.com/story/show/51717513
Chris Beer finished "Within a transaction, graph subjects should include the transaction identifier for all node-subjects." https://www.pivotaltracker.com/story/show/5171751319:45
<cbeer>awoods: i made you the reviewer on that. if we see ajs6f tomorrow, you could try to pawn it off on him19:53
<bljenkins>Project fcrepo-fixity-corrupter build #118: SUCCESS in 1 min 15 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity-corrupter/118/19:58
<awoods>cbeer: ok. I believe ajs6f is back online Wed.20:00
<bljenkins>Project fcrepo-fixity build #302: SUCCESS in 4 min 52 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/302/20:01
Project fcrepo-kitchen-sink build #397: SUCCESS in 5 min 16 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-kitchen-sink/397/20:03
<cbeer>awoods (eddies?): do you know how to dump all the jmx statistics from the command line?20:42
(without writing a bunch of java code to do it..)
<awoods>cbeer: you mean the log output?20:43
which "statistics" are you referring to?
<cbeer>awoods: the stuff in this image https://wiki.duraspace.org/download/attachments/34654056/fcrepo-metrics-jmx.png?version=1&modificationDate=137090141648620:44
<awoods>oh, those jmx
for some reason, jmeter uses the jmx extension as well.20:45
cbeer: sorry, I do not know off hand
<cbeer>awoods: np. i've only connected to them through a gui or dumped them by name20:46
* nbanks joins21:17
* nbanks leaves21:22
<pivotal-bot>Chris Beer added "Do some logging and see what's taking MODE/ISPN so long to save sessions after lots of objects are in the repository" https://www.pivotaltracker.com/story/show/5186404321:40
* nbanks joins23:18
* nbanks leaves23:22

Generated by Sualtam