Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* ksclarke leaves02:24
<pivotal-bot_____>Ye Cao started "Wire and integration test Solr indexer into webapp" https://www.pivotaltracker.com/story/show/5745328003:54
Ye Cao added comment: "Init integration test for solr Indexer is added ( SolrIndexerIT.java)04:01
TODO:
1. currently manually comment stand..." https://www.pivotaltracker.com/story/show/57453280
Ye Cao deleted "Create Solr unit tests" https://www.pivotaltracker.com/story/show/5848290804:13
Ye Cao deleted "Search" https://www.pivotaltracker.com/story/show/5715317604:14
Ye Cao deleted "Create follow-on Solr tickets" https://www.pivotaltracker.com/story/show/58341334
* fasseg joins04:34
* fasseg leaves05:12
* fasseg joins05:13
hmm it seems im carrying some state from a previous fedora4 session in an image file which is used by the VM to run nodes, that's why im getting those inconsistency errors i mentioned the day before yesterday07:56
<pivotal-bot_____>Frank Asseg added "Test different confguration settings of infinispan/jgroups in order to raise performance on the SCC cluster" https://www.pivotaltracker.com/story/show/5940135008:36
* ajs6f joins
<pivotal-bot_____>Frank Asseg started "Test different confguration settings of infinispan/jgroups in order to raise performance on the SCC cluster" https://www.pivotaltracker.com/story/show/59401350
Frank Asseg finished "Run benchmarks on SCC cluster at FIZ Karlsruhe with fcrepo3 and fcrepo4" https://www.pivotaltracker.com/story/show/52289545
* ajs6f1 joins08:46
* ajs6f leaves08:47
<fasseg>hmm i get 15mb/s more throughput on a makeshift 4 node cluster at home on which two nodes are connected via wifi, than on the SCC cluster :/08:49
* osmandin joins
<ajs6f1>WTF?
<fasseg>in my head the SCC cluster slowly takes the form of just a couple of phones08:50
and that's about 200% througput, since i have around 8 mb/s on the cluster!08:51
<ajs6f1>Have you checked the home-grown cluster against fcrepo3?08:52
<fasseg>*checkout my 15 node android cluster with bluetooth multicasting"
nah
<ajs6f1>I don't feel like we have a reliable sense _yet_ of the throughput for fcrepo4, and I don't thikn we're going to get it anytime soon. I think we can try at best for a "proportion" between fcrepo4 and fcrepo3.08:53
<fasseg>that's right f4 throughput is very volatile for me, very much depending on I/O and networking throuput of the cluster08:54
<ajs6f1>Maybe gregjansen or tecoripa's ideas about using local clusters can help...08:55
Their local clusters, that is.
<fasseg>Im thinking about connecting my 2 leftover nodes at home by wire and run a test, but ill ask awoods first if I should invest the time...08:56
it involves physical transportation of nodes ;)
<ajs6f1>I think we need to plan the cluster testing as a group a little better, instead of just sending you out to make your way through the woods.08:57
<fasseg>well osman is joinin the clustering work
<osmandin>fasseg: i tested your benchmarks, and it seemed it was still taking 40 minutes or so
<fasseg>so we might have some feedback from him soon..
oi
have you by any chance already checked the network and I/O performance of the hardware?08:58
that's even worse throughput than on SCC cluster right? <8mb/s
<osmandin>did you have any notes for any modified settings.e.g. for jgroups.. or r your number out of box09:00
<pivotal-bot_____>A. "Brulestion" Soroka added "Code cleanup for fcrepo-http-*" https://www.pivotaltracker.com/story/show/5940295409:03
A. "Brulestion" Soroka started "Code cleanup for fcrepo-http-*" https://www.pivotaltracker.com/story/show/59402954
A. "Brulestion" Soroka added comment: "https://github.com/futures/fcrepo4/pull/120" https://www.pivotaltracker.com/story/show/59402954
A. "Brulestion" Soroka accepted "Code cleanup for fcrepo-http-*" https://www.pivotaltracker.com/story/show/5940295409:04
<fasseg>osmandin: not reall notes no, first I used the normal clustered/infinispan.xml config, than I tried this one with leveldb:
<pivotal-bot_____>A. "Brulestion" Soroka finished "Code cleanup for fcrepo-http-*" https://www.pivotaltracker.com/story/show/59402954
<fasseg>https://gist.github.com/fasseg/7118316
osmandin: and then I will try with write-behind fSyncMode in infinspan's FileStoreCache as described here: https://docs.jboss.org/author/display/ISPN/Write-Through+And+Write-Behind+Caching09:06
<osmandin>thanks..
<fasseg>I also want to try to enable compression in jgroups: which is something like <COMRESS compression_level=[0-9]/> check e.g. here: http://www.jgroups.org/javadoc/org/jgroups/protocols/COMPRESS.html09:07
but at the moment im having trouble gettin a cluster at SCC up09:08
<osmandin>thanks. for hdd my numbers (copying bytes to /zero) were around 100-125mb/s09:09
<fasseg>nice...I only got 40mb/s since it's a mounted partition :/
and therefore decreases available network I/O :(09:10
* gregjansen joins09:14
* github-ff joins09:15
[fcrepo4] ajs6f pushed 1 new commit to ExposeRdfStream: http://git.io/SZK7Mg
fcrepo4/ExposeRdfStream bc8cf50 ajs6f: Renamed GraphSubjects to IsTranslator
* github-ff leaves
<ajs6f1>Oops. That comment should have read "IdTranslator"
Anyone know how to edit a comment if you've already pushed?09:16
<fasseg>--amend and push again?
<ajs6f1>I'll give that a try
<mikeAtUVa>You have to force the push, it's a rewrite of history.
<fasseg>you can also recommit and force push
<ajs6f1>thnx
* github-ff joins09:17
[fcrepo4] ajs6f force-pushed ExposeRdfStream from bc8cf50 to 0d8d13c: http://git.io/dPEWUA
fcrepo4/ExposeRdfStream 0d8d13c ajs6f: Renamed GraphSubjects to IdTranslator
* github-ff leaves
<ajs6f1>fasseg++09:18
mikeAtUVa++
<fasseg>I get 15 sec timeouts now: https://gist.github.com/fasseg/711861409:20
it's getting depressing
<osmandin>I was getting the same, but only on one node09:25
* fasseg leaves
<ajs6f1>afk bbs
* ajs6f1 leaves09:26
* fasseg joins09:30
* ermadmix joins09:38
* ksclarke joins09:45
<pivotal-bot_____>Andrew Woods added "UWisc Cluster Request" https://www.pivotaltracker.com/story/show/5940843810:00
Andrew Woods edited "UWisc Cluster Request" https://www.pivotaltracker.com/story/show/59408438
<bljenkins>Yippie, build fixed!10:02
Project fcrepo4 build #1216: FIXED in 25 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1216/
Project fcrepo-fixity-corrupter build #293: SUCCESS in 1 min 48 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/293/10:04
Project fcrepo-kitchen-sink build #545: UNSTABLE in 3 min 47 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/545/10:08
Project fcrepo-fedora3-federation-connector build #70: UNSTABLE in 5 min 46 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fedora3-federation-connector/70/
* ajs6f joins10:11
<pivotal-bot_____>Frank Asseg added comment: "Full Name:10:17
Frank Asseg
Preferred Email:
frank.asseg@fiz-karlsruhe.de
Net ID:
n/a
Department: ""
Electroni..." https://www.pivotaltracker.com/story/show/59408438
Frank Asseg accepted "UWisc Cluster Request" https://www.pivotaltracker.com/story/show/59408438
Frank Asseg started "UWisc Cluster Request" https://www.pivotaltracker.com/story/show/5940843810:18
<awoods>fasseg: You are not wasting any time.
<fasseg>awoods: I finished my infos on the ticket so you can grab it again
<awoods>thanks, fasseg.
<fasseg>the lookout to get anouther cluster to run fcrepo4 on sped things up ;)10:19
like i Said im getting 15 secs timouts atm on the SCC VMs :/
<awoods>It looks like that error has some google hits.
btw, fasseg, have you been able to set up a cluster that improves throughput with more nodes (behind a load-balancer)?10:20
<fasseg>at home yes, at SCC there was almost no positive effect to loadbalancing
I just did a simple roundrobin loadbalancer via mod-jk and apache10:21
<awoods>and the more nodes, the better throughput?
<fasseg>that's part of the scc scripts in the repo
nope...
the more nodes almost the same throughput since I hit other bottle necks on scc10:22
at home yes
<awoods>It sounds like we need another cluster... scc seems to have other issues.
<fasseg>from 2 to 4 nodes i get ~36% throughput increase but the last two nodes are on a wifi
<awoods>hopefully UWisc will work out.10:23
<fasseg>I got mail from the admin today that he will take a look tomorrow
<awoods>Was there also a UNC cluster option from gregjansen?
<fasseg>since there are a lot of machines running for more than 100 days with 99cpu load atm
<awoods>fasseg: Would you be interested in working through the cluster issues on AWS?10:25
<gregjansen>I haven't gotten a response about the cluster yet, awoods
<awoods>gregjansen, How long to you expect for a response?10:26
<fasseg>I can certainly try but I have no experience on AWS whatsoever
* tecoripa joins
<awoods>fasseg: I have most of it scripted in https://github.com/futures/fcrepo-aws-puppet
fasseg: We can chat after the standup10:27
<fasseg>awoods: give me one more day on the SCC cluster and if the admins can't reduce the load by tomorrow..go on AWS?
ok cool
* ksclarke leaves
<awoods>fasseg: deal
<fasseg>k
<awoods>All: Here is the agenda-placeholder for tomorrow's committer call. Please update.10:29
https://wiki.duraspace.org/display/FCREPO/2013-10-24+-+Fedora+Committer+Meeting
<gregjansen>awoods: this week10:32
<awoods>gregjansen: What did you actually request?
<barmintor>awoods: did you see my emal from last night?
<awoods>barmintor: Yes, I did.10:33
<pivotal-bot_____>Ye Cao added comment: "TODO: at end of wiki page:https://github.com/futures/fcrepo-jms-indexer-pluggable/wiki#running-the-indexer10:34
descri..." https://www.pivotaltracker.com/story/show/57453326
Ye Cao started "HTML description of Solr setup" https://www.pivotaltracker.com/story/show/57453326
<awoods>barmintor: It sounds like good status and a good plan... although getting updates in FCSU would be nice.
<barmintor>awoods: they're completely independent releases, though, so I would rather get the bugfix out10:35
<awoods>barmintor: definitely
* fasseg leaves10:36
<awoods>barmintor: Should we hold another meeting on this bug?
<barmintor>awoods: No.
* fasseg joins
<barmintor>not unless you want feedback about announcements, but that can happen asynchronously10:37
<awoods>barmintor: Have you spoken with mikeAtUVa?
<barmintor>yes
some of his ffedback went into the patch
<pivotal-bot_____>Ye Cao added comment: "Input will be at https://wiki.duraspace.org/display/FF/solrIndexerSetup" https://www.pivotaltracker.com/story/show/5745336010:39
Ye Cao started "Create user-facing documentation for Solr setup" https://www.pivotaltracker.com/story/show/57453360
<barmintor>mikeAtUVa suggested the log message re: "inconsistent state" in the patched DefaultDOManager10:40
(thanks mikeAtUVa)
<pivotal-bot_____>Ye Cao added comment: "Input will be at https://wiki.duraspace.org/display/FF/SolrQueryRecipes" https://www.pivotaltracker.com/story/show/57453388
Ye Cao started "Solr query recipes" https://www.pivotaltracker.com/story/show/57453388
* fasseg leaves10:42
* fasseg joins10:48
<gregjansen>tecoripa: I found the root cause for the halting integration test10:51
tecoripa: it was an exhausted connection pool in the apache http client..
<osmandin>fasseg have you tried mass deleting your test objects? i get replication timeouts when using REST10:52
<tecoripa>gregjansen: okay. did you commit a fix?
<fasseg>osmandin: no I have indeed not
but deleteing the top level object /rest/objects worked for me always10:54
via the html interface
<barmintor>awoods: #duraspace-ff-bug
<ajs6f>cbeer: ping?
<osmandin>fasseg: thanks, i can get rid of one's node's objects but it seems to be taking up forever for propagation10:59
<gregjansen>tecoripa: I have pushed a fix
<fasseg>hmm strange this was almost instantly for me
<tecoripa>gregjansen: great. I'll update my workspace. I still need to finish configuring your project so it builds.11:00
<gregjansen>tecoripa: ready to help you out on that front
<tecoripa>gregjansen: thanks, I may reach out11:01
<awoods>standup: ermadmix
osmandin
<osmandin>fasseg: it's funny it deleted on the 1/2, and now the objects are back!11:02
<fasseg>hmm that sounds bad...11:03
lemme try to reproduce as soon as the cluster is up...
<cbeer>ajs6f: pong11:06
<ajs6f>cbeer: I'm banging away at this RDF iteration thing...
cbeer: and I have gotten to the point of engaging with the domain objects, e.g. FedoraResource.11:07
cbeer: I'm trying to understand one simple thing:
cbeer: When a FedoraResource is updated, e.g. via
createOrReplaceObjectRdf() or updateSparql()...
cbeer: How/where are the changes persisted back into the JCR?11:08
cbeer: I only see RDF being manipulated...
I have a felling I'm missing something obvious.
<cbeer>ajs6f: magic. scary black magic. one sec
<ajs6f>O! np then. I'll just migrate to use sorcery.
<pivotal-bot_____>Osman Din added comment: "https://gist.github.com/osmandin/7120547" https://www.pivotaltracker.com/story/show/58726512
Ye Cao added comment: "Also at index.html of the webapp" https://www.pivotaltracker.com/story/show/57453326
<cbeer>https://github.com/futures/fcrepo4/blob/master/fcrepo-kernel/src/main/java/org/fcrepo/kernel/utils/JcrPropertyStatementListener.java maybe?11:09
<ajs6f>OOOOOHHHHHH....
<cbeer>somewhere it gets registered as a listener
<ajs6f>Yes, I forgot about that guy.11:10
Okay, I'll have to introduce a totally new pattern. That's cool.
<cbeer>very nice, because we can do whatever to manipulate the Jena graph and have it magically appear
but certainly non-obvious
<ajs6f>Yeah, that is nice. I'm going to have to be a lot more explicit… or maybe I can buy some of that magic. Have to see.11:11
I'll probably have to be more explicit.
<fasseg>tecoripa: https://github.com/futures/benchtool there's a small Readme which explains the usage...
<tecoripa>fasseg: thanks. got it.11:12
<cbeer>ajs6f: good luck. i think i had to choose this pattern in order to reasonably use the sparql update stuff jena provided11:13
<osmandin>https://wiki.duraspace.org/pages/viewpage.action?pageId=3466802711:14
<ajs6f>cbeer: yeah, I get that. I'm expecting to have to move through in-memory Jena Models for the SParql stuff, but that's not too big a hit. It was the simple GETs that were killing mikeAtUVa.11:15
<cbeer>ajs6f: btw.. at DLF, can you show me some stuff about Stanbol? I don't think I did a good job convincing people that the-product-i've-never-been-able-to-understand-what-it-actually-does solves all our problems and is better than rolling their own thingy11:17
<ajs6f>Sure.
One of the problems with talking about Stanbol is that it isn't an application designed for a particular need. It's a set of components designed to support many different cases, depending on how they are wired and integrated. They aren't expected to much of anything interesting on their own. They also assume (on the part of the integrator) a pretty deep and wide knowledge of SemWeb tech.11:19
<barmintor>Stanbol: It's the Fedora of RDF
<ajs6f>Fedora: It's the Stanbol of bits.
Folks who don't customarily think in terms of architectures larger than a single app or who aren't particularly versed in the SemWeb stuff find Stanbol hard to grok.11:20
Ditto Fedora.
<barmintor><- one o' them folks
<ajs6f>That's why you hired me.11:21
<pivotal-bot_____>Frank Asseg added "Add configuration files used to the wiki SCC benchmark results for better reproducability" https://www.pivotaltracker.com/story/show/5941536011:22
<ajs6f>awoods: Just a ping, this: https://www.pivotaltracker.com/story/show/5940295411:29
is done.
<pivotal-bot_____>feature: Code cleanup for fcrepo-http-* (finished) / owner: A. "Senphitic" Soroka
Osman Din started "Document performance numbers in wiki" https://www.pivotaltracker.com/story/show/5935984011:31
Osman Din started "Create fcrepo4 single node benchmarking result on the Yale cluster" https://www.pivotaltracker.com/story/show/5925284011:32
* ksclarke joins11:33
<pivotal-bot_____>Scott Prater added comment: "I'll take care of the contact details, PI details. What I need to know are more the technical details of w..." https://www.pivotaltracker.com/story/show/5940843811:39
Gregory Jansen added "Support changing the roles assigned at repository root" https://www.pivotaltracker.com/story/show/59417046
Scott Prater added "Fedora performance tests with/without active basic roles and PEP enforcement" https://www.pivotaltracker.com/story/show/5941731411:43
<cbeer>ajs6f: i think i mentioned earlier, one of the things that came up at the meeting this week (notes to come, btw.. we're actually editing the notes for readability or something) was the whole subproperties thing11:47
i did some digging into the LDPath Jena backend, which does support inferencing and reasoning, i guess11:48
(and, i looked at our less-great choices in ruby-land, none of which seem to support reasoning off of an owl/rdfs/etc doc)
<pivotal-bot_____>Andrew Woods added comment: "Does this give you enough, @scottprater? At the very least, I suspect we need to provide a RAM number." https://www.pivotaltracker.com/story/show/5940843811:49
<ajs6f>It does, but what is "subproperties"? You mean in the subsumption sense/
?
myProperty < yourProperty
?
<pivotal-bot_____>Andrew Woods edited "Add configuration files used to the wiki SCC benchmark results for better reproducability" https://www.pivotaltracker.com/story/show/59415360
Andrew Woods edited "Add configuration files used to the wiki SCC benchmark results for better reproducability" https://www.pivotaltracker.com/story/show/59415360
<cbeer>ajs6f: sure. via rdfs:subPropertyOf or otherwise
<pivotal-bot_____>Andrew Woods edited "Create fcrepo4 single node benchmarking result on the Yale cluster" https://www.pivotaltracker.com/story/show/5925284011:50
Andrew Woods edited "Create fcrepo4 single node benchmarking result on the Yale cluster" https://www.pivotaltracker.com/story/show/59252840
<ajs6f>cbeer: Gotcha.
<cbeer>ajs6f: so.. i'm wondering how crazy or non-performant it might be to (optionally, of course) have jena do some reasoning for us and tack those triples onto the in-memory graph11:51
<ajs6f>cbeer: So are we going to bust out the OWL Full craziness?
cbeer: It really depends. For RDFS, probably not too bad.
At least not over big data.
<cbeer>ok. let's say RDFS only, then.
<ajs6f>cbeer: That's a good place to start.11:52
<cbeer>and, maybe, we could just store some RDFS documents in some well-known location in the JCR tree?
<ajs6f>cbeer: The problem with OWL is not that it's necessarily slow or bad, it's that the tractability is not as tightly bounded.
Even using the profiles.
cbeer: I don't know about where to put it… sure, maybe establish a privileged section of repo to begin with.11:53
<cbeer>and poof, magic happens, consuming apps get all the triples they deserve without having to work too hard at it (and, presumably, we could cache the reasoned triples somewhere for safe keeping)
<ajs6f>Or maybe a priviledged mixin type that carries RDFS assertions?
<awoods>cbeer: Will you be available to give a brief overview of the Hydra/RDF workshop outcomes at tomorrow's Fedora committers meeting?
* osmandin leaves
<ajs6f>cbeer: For performance, we'd really have to. We're talking about bringing a triplestore back into the core, but as long as we're taling about a cool optional component, I'm down with that.
<cbeer>ajs6f: do we really need a full triple store (ok, yes, probably.. if we want to do a thorough job).. i was hoping to get away with the jena in-memory model11:54
<ajs6f>cbeer: There's a totally different way to approach this: do the asynch indexing into an external 3store (as is being worked on nowish) and inference the hell out of the data there.11:55
<cbeer>i guess we'd have to define our actual usecase.
<ajs6f>cbeer: But this is Fedora! We never do that!
<cbeer>awoods: i can try. i'm not sure there was much that was actually relevant to us
yet, at least.
<awoods>cbeer: so it may be very brief.
<cbeer>there was some good fcrepo4 Q&A, and tcramer had some good notes on that11:56
<awoods>cbeer: Those notes would be good to see
<cbeer>ajs6f: so, you didn't laugh at me (publicly). once you finish your rdf refactor, maybe i'll play around with this some more11:57
awoods: to come. i think we said by the end of the week. certainly no later than DLF
<ajs6f>It sounds like fun. We have at least two interesting directions: Stanbol <-> Fedora integration, and inferencing inside the repo.11:58
<awoods>cbeer: btw, when are you "back on the sprint"?
<cbeer>awoods: now-ish. i'm trying to get up to speed as we speak11:59
<ajs6f>cbeer: It's much as you left it, except we changed to writing APL.
<awoods>cbeer: Great. Maybe we can get on google-hangout later today to discuss.
* github-ff joins12:01
[fcrepo4] ajs6f pushed 1 new commit to ExposeRdfStream: http://git.io/ZB6GuA
fcrepo4/ExposeRdfStream 72aa4d8 ajs6f: RdfStream can now be created from Model
* github-ff leaves
<ajs6f>afk bbs12:02
<cbeer>mikeAtUVa: ping?12:07
<mikeAtUVa>cbeer: yup
<cbeer>mikeAtUVa: i'm just going through tickets trying to figure out where things stand now and https://www.pivotaltracker.com/story/show/57812960 caught my eye12:08
<pivotal-bot_____>feature: Research and design the REST semantics for object (and tree) copying (unscheduled) / owner: Mike Durbin
<cbeer>that's a fcrepo3 => 4 ticket because we want to be able to say something like "move all these fcr3 objects into fcr4 and into modeshape proper, not just as a projection"?
<awoods>cbeer: correct12:09
<mikeAtUVa>Yeah, though I think we could write it generally enough so that we can copy any tree (node and descendants) from anywhere (readable) to anywhere (that's writable)
<cbeer>mikeAtUVa: sounds good. i thought about this some on another ticket too... seems like we have a couple choices.12:10
maybe pick up the WebDAV move/copy verbs, maybe not the end of the world
or add a POST endpoint with some source and destination headers (custom thing, but at least not an abuse of HTTP?)12:11
or do something like CMIS (shudder) and create some payload that says "move this stuff over there"
i'd be inclined towards #2 (e.g. POST /fcr:(move|copy|clone)/some/path/to/a/node)12:13
but there's something really nice about the WebDAV verbs too
<mikeAtUVa>cbeer: I think simpler is better, and some reasonable handling of known fedora node types12:14
The post at fcr:move, copy clone seems to fit that bill..
cbeer: I'm maybe not the best person to weigh in on the style decisions here, as long as the API is functional and not too cumbersome, I'm happy.12:15
<awoods>mikeAtUVa: All input is valued. I like the RESTful-ish approach.12:16
<mikeAtUVa>cbeer: I have no problem with reuse of well-known semantics so long as the model is reasonably similar. I figure our copy should not be allowed to copy an fedora:object into a fedora:object, or a fedora:datastream... but wouldn't that be supported by restrictions in the node types?12:17
<cbeer>ok. no one said "WebDAV. cool!"
mikeAtUVa: yeah, i think that makes sense. make JCR do the validation for us
<mikeAtUVa>cbeer: I don't know if those rules are in place now (or can you even define in JCR what child types are allowed?)12:18
<cbeer>mikeAtUVa: yep, you can, in the CND file
doesn't look like we have those rules in place (and, maybe we don't need them.. does any harm come to anything by allowing it?)12:19
<ajs6f>WebDAV. cool!12:20
cbeer: Seriously, if we want to be RESTful, the WebDAV extension is better than the other choice.
Verbs should be verbs.
<cbeer>ajs6f: ok. there it is. i kinda have the same reaction12:21
ajs6f: and then go "WebDAV? does anyone use WebDAV? does e.g. the PHP REST client even know how to handle things like that?"
<ajs6f>cbeer: Are we RESTafarian, or just a bunch of potheads?
<mikeAtUVa>cbeer: those rules aren't in place, but I think they should be if (when) we've decided that everything in our fedora 4 should be fedora objects.
<fasseg>shouldn't we rather provide such functionality based on the existing rest endpoints in a dedicated client, or the java api? i would hesitate to blow the HTTP api up...
<cbeer>i do like COPY /some/path/to/a/ode (Request-URI: /put/the/node/here)
<ajs6f>cbeer: Actually, I think we're a lot more likely to find WebDAV clients than Fedora clients. {grin}
<bljenkins>Project fcrepo4 build #1217: UNSTABLE in 21 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1217/12:22
<awoods>Here is an HTTP Copy example: https://wiki.duraspace.org/display/DURACLOUDDOC/DuraCloud+REST+API#DuraCloudRESTAPI-CopyContent12:23
<ajs6f>cbeer:awoods: What do you think of this replacement for the persisting-listener: a persisting iterator<triple>. You wrap your iterator<triple> in this guy, and then as you pull on him (the persisting-iterator), he not only gives you triples from the inner iterator, he persists them as they come through…
<awoods>ajs6f: Does that mean that updates are not persisted until they are read?12:24
<mikeAtUVa>Do we expect the basic features of fedora 4 will just work for people using WebDAV or CMIS or whatever, and only those functions that go beyond the scope of those applications should be in the fedora API?
<awoods>mikeAtUVa: Having a hybrid interaction model (somethings you do with WebDAV and others you do with fedora API) seems a little dirty.12:26
<cbeer>ajs6f: i'm not sure i understand. where is iterator<triple> coming from?
<ajs6f>awoods: No, no, definitely not. It's just a question of where you put the persistence. In this case, you might wrap some iterator within a tree of them (such as we are now constructing) with this new guy.
cbeer: We've move from passing Jena Models and Datasets around to passing around Iterator<Triple> (actually our own class RdfStream impls Iterator<Triple>) for scaling reasons.
cbeer: Nodes with huge numbers of children were blowing out heaps.12:27
cbeer: Because the Jena constructs were in-memory.
<awoods>ajs6f: It is still not clear how you are envisioning the persistence tying into an update flow.12:28
<ajs6f>cbeer: RdfStream can produce a Model from itself (for serialization or SPARQL stuff) and a Model can turn into an RdfStream via a static method.
cbeer: But if you stay with RdffStream, you never pull all the triples into heap at once.
<cbeer>ajs6f: with you so far.12:29
<ajs6f>awoods: Take a look at replacePropertiesDataset
in FedoraResource.
That's doing the same thing, but with Models.
(And the listener, who comes in invisibly.)12:30
cbeer: So we can use pure iteration for things like GET on a node, but we do SPARQL Update (say) by going to a Model and back.
cbeer: My hope is that no non-mutating operation would require going to Model.12:31
<cbeer>ajs6f: ah, i thought you were trying to suggest this for updatePropertiesDataset and the sparql-update stuff
yes, i think a persisting triple iterator would work fine for the replacePropertiesDataset
you might need two variations, though.. one to remove properties, one to add properties12:32
<ajs6f>cbeer: Mm… good point. Let me think about it a little harder.
<cbeer>(as long as we ignore the possibility of blank nodes, that is.)
<ajs6f>&T^&**T*T BLANK NODES!
And you can tell Declan that.
{grin}
cbeer: Seriously, the whole Iterator<Triple> thing definitely screws up on blank nodes. I'm not troubled by that right now.12:33
<cbeer>i don't blame you (because it isn't the only thing it screws up)12:34
<ajs6f>If we have to, we can attached context to RdfStream to deal with that.
<cbeer>just being a rabblerouser.
<ajs6f>I attached a namespace map to RdfStream to help with seriazliation, and if people _really_ want blank nodes, then 1) They're wrong, and 2) We'll do it by attaching more context to RdfStream.
Cute: http://download.java.net/jdk8/docs/api/java/util/Spliterator.html12:38
Not immediately useful to us, but cute.
awoods; When do we move to Java8?
<awoods>ajs6f: not on the docket, right now.12:40
<ajs6f>awoods: So Scala, then? Cool.
<pivotal-bot_____>Chris Beer started "Research and design the REST semantics for object (and tree) copying" https://www.pivotaltracker.com/story/show/5781296012:42
Chris Beer started "Implement node/tree copy." https://www.pivotaltracker.com/story/show/57813074
<cbeer>mikeAtUVa: one of the implications of whole-repository-copy i didn't consider was the ability to make that asynchronous
i'm kinda tempted to punt on that and defer to some "make the REST API asynchronous" ticket for that12:43
<ajs6f>cbeer: It doesn't even have to be the whole shebang. A big section of repo (or a few objects with really huge datastreams) could want async.12:53
<cbeer>ajs6f: sure, of course.12:54
<tecoripa>all: some thinking on exceptions and logging of AuthZ errors: https://wiki.duraspace.org/display/FF/Basic+Java+Authorization%3A++Exceptions+and+Logging12:56
one of my pet peeves with fedora 3 is this difficulty finding out what caused an Authorization denied error
I want to try to nip that problem in the bud right from the get.12:57
* fcrepo-bot joins
<tecoripa>that page is the result of a discussion Greg and I had after the standup...12:58
<ajs6f>http://www.youtube.com/watch?v=HqEIt-rQqbM
<tecoripa>I'd like to test these exceptions in the integration tests NOW! Nip it in the BUD!12:59
<cbeer>ajs6f: jax-rs question for you.. so, i'm doing this copy/move thing, hoping it'll jog some memory of how to code in java13:00
<tecoripa>anyways, comments, constructive (or snarky) criticism welcome
<cbeer>ajs6f: the webdav spec says the destination will be provided in a header as a full URI
<ajs6f>k
<cbeer>e.g. Destination: http://www.ics.uci.edu/users/f/fielding/index.html
ajs6f: somehow i need to translate that into a JAX-RS path, i guess. but we've only built out infrastructure in http-commons for handling JAX-RS path segments, it seems13:01
<ajs6f>k
Wait, don't you need to translate into a _JCR_ path?
<cbeer>yes, sorry.
<ajs6f>k
<cbeer>either i need to go from that string to a List<PathSegments> (and, that may not be enough) or i need to refactor http-commons to have some string-aware parsing13:02
<ajs6f>Hm. Doesn't HttpGraphSubjects do this?
<cbeer>ooh, maybe it does
<ajs6f>h/t to barmintor for thinking of this moment when he wrote that class in the spring. The man has foresight.13:03
<cbeer>yeah, i can turn the string into a resource
except we punted on the "figure out cross-workspace" thing.
but that's easy to fix
thanks
<ajs6f>rock on
<cbeer>(s/easy to fix/easy to punt on still/)
<ajs6f>punt on
<barmintor>did I do something right? That feels kind of novel these days.13:04
<ajs6f>fcrepo3 will make you feel that way.13:05
<barmintor>Fedora 3.7 allows you to rebuild your sense of shame from the command line13:06
<pivotal-bot_____>Chris Beer added "Refactor (to DRY out) the HTTP path => JCR path logic" https://www.pivotaltracker.com/story/show/5942450813:07
Scott Prater added comment: "A few details maybe you and Frank could nail down: ""13:08
Question about CPU intensive operations -- calculating f..." https://www.pivotaltracker.com/story/show/59408438
* tecoripa leaves13:11
<ajs6f>cbeer: Just a double check— it looks like our contract now is that if you send us triples with subject that aren't Fedora stuff, we just ignore them. Does that sound right?13:12
// if it's not about a node, ignore it.
if (!subjects.isFedoraGraphSubject(subject)) {
return;
}
)that's from the listener)
* kaarefc joins
<cbeer>ajs6f: yep, sounds right.
<ajs6f>cbeer: Cool.
<cbeer>ajs6f: there's some on-going work in LDP to figure out how to report that kind of error back to the consumer
but it isn't fatal.
<ajs6f>cbeer: Cool. I'll stay with our contract now, and we can move to mee LDP when it gets somewhere.13:14
<pivotal-bot_____>Frank Asseg added comment: "@scottprater: ""13:23
RAM:
My current VMs run with 4Gb, so thats enough, of course 8gb would be nicer to have some m..." https://www.pivotaltracker.com/story/show/59408438
* osmandin joins13:29
<pivotal-bot_____>Frank Asseg added comment: "This is the an example of the CPU load I see on the VMs during ingest: ""13:35
" https://www.pivotaltracker.com/story/show/59408438
Scott Prater added comment: "Thanks, Frank. I have enough info now to craft and submit the request.13:40
Once the request is submitted, an i..." https://www.pivotaltracker.com/story/show/59408438
Andrew Woods added comment: "Thanks @scottprater and @frankasseg." https://www.pivotaltracker.com/story/show/5940843813:42
Scott Prater added "Submit request to use University of Wisconsin - Madison cluster hosted on their Advanced Computing Infrastructure" https://www.pivotaltracker.com/story/show/5942778413:44
* tecoripa joins
awoods: I thought about putting the exceptions and logging notes on the access role modules page, but decided against it, as the exceptions and logging are really under-the-hood technical details, whereas the access role module is higher-level documentation, which may even make it (in an altered form) into user dpocumentation at some point.13:46
<cbeer>" invalid target release: 1.7" means mvn is picking up the wrong version of java, right?13:47
<awoods>cbeer: yes, most likely.
tecoripa: sounds reasonable.
* github-ff joins13:50
[fcrepo4] ajs6f force-pushed CleanupHTTPModules from 999c9cd to bb068a2: http://git.io/G-dX9A
fcrepo4/CleanupHTTPModules ee816d1 ajs6f: Code cleanup including whitespace adjustment, static imports, and removing unneeded type-safety annotations
fcrepo4/CleanupHTTPModules f00d961 ajs6f: More static imports, whitespace cleanup, etc.
fcrepo4/CleanupHTTPModules bb068a2 ajs6f: Fixing indentation errors, etc.
* github-ff leaves
<ajs6f>awoods: ^^^
* tecoripa leaves13:52
<awoods>ajs6f: on a call13:53
<ajs6f>NO!13:54
{/snark}
afk bbs
* ajs6f1 joins13:58
* ajs6f leaves13:59
* gregjansen leaves14:04
* fcrepo-bot leaves14:05
* ajs6f1 leaves14:07
<bljenkins>Yippie, build fixed!14:15
Project fcrepo4 build #1218: FIXED in 25 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1218/
Project fcrepo-fixity-corrupter build #294: SUCCESS in 1 min 22 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/294/14:17
* ajs6f joins
<bljenkins>Project fcrepo-kitchen-sink build #546: STILL UNSTABLE in 3 min 15 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/546/14:20
* jonathangee leaves14:24
<awoods>ajs6f: CleanupHTTPModules push coming in just a moment.
ajs6f: thanks for the rebase.14:25
<ajs6f>Sorry for the hiccup. I hadn't realized what it meant that you had merged the RDF stuff already.
I'll try to keep better track.
* mikeAtUVa leaves14:37
<ajs6f>cbeer/awoods/et al.: Using the Jena types, we've been keeping differently-sourced triples in different Models inside Datasets. (e.g. version-related triples, property-related triples, etc.)
In the iterating modality, we don't.
* ksclarke leaves
<ajs6f>Assuming that we can provide iterators of these different kinds of triples when we want,14:38
<cbeer>ajs6f: i'm not sure it's important. i think it was just to make the updating actions easier
<ajs6f>which we can, do you see any problem with the default being an undistinguished stream? IOW, if you just ask a resource for its triples, you'll get all of them?
cbeer: thx
My concern is indeed re: updating: I don't want to accidentally record triples relating to versioning as properties, that kind of thing.14:39
I guess I'll have to keep track of different "classes" of triples.
cbeer: There's an internal use case for inferencing!
<cbeer>yep. that's it, exactly.
i think we can exclude most of them by predicate
<ajs6f>Yes.14:40
I'd like to rely on a true ontology, but I'll rely on sets of predicates or something like that.
cbeer: You made RdfLexicon to keep track of this kind of thing, right? I'll put some machinery in there to help with this.14:41
<awoods>ajs6f: sounds like you have it in-hand.14:42
<ajs6f>Well, it's kind of gooey and it runs out through your fingers when you squeeze it, but more-or-less.
* gregjansen joins
* ksclarke joins14:44
* github-ff joins
[fcrepo4] awoods pushed 1 new commit to master: http://git.io/RmXz3Q
fcrepo4/master d08914b ajs6f: Non-functional reformatting update...
* github-ff leaves
<pivotal-bot_____>Andrew Woods added comment: "Resolved with: https://github.com/futures/fcrepo4/commit/d08914b0aa5f07a5b600a5049bd6b6435a6009b4" https://www.pivotaltracker.com/story/show/5940295414:45
Andrew Woods delivered "Code cleanup for fcrepo-http-*" https://www.pivotaltracker.com/story/show/59402954
Osman Din started "Create fcrepo4 multi node benchmarking result on the Yale cluster" https://www.pivotaltracker.com/story/show/5925305014:52
Osman Din added comment: "Compare documentation for comparison: https://wiki.duraspace.org/pages/viewpage.action?pageId=34668027" https://www.pivotaltracker.com/story/show/5925305014:53
<cbeer>ajs6f: so i'm extracting the uri => jcr path stuff into a separate method (because the existing method returns an existing node, not a path)14:55
<ajs6f>Wait, in HttpGraphSubjects?14:56
<cbeer>yes
<ajs6f>k
<cbeer>but we were (in part) relying on testing whether the node exists to tell us if the thing actually was a valid jcr path
(so, if it is, return the object, if it isn't, return null.. just like the logic you ran into earlier, i guess)
<ajs6f>Well, if you're trying eventually to find a node with that path, isn't that okay?
* travis-ci joins
[travis-ci] futures/fcrepo4#1045 (master - d08914b : ajs6f): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/99564d96fdea...d08914b0aa5f
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/12947029
* travis-ci leaves
<ajs6f>Oh, wait, there might not be a node there yet...14:57
<cbeer>perhaps. i guess there's no harm in asking to move(/some/path, /some/invalid?bad-path)
<ajs6f>cause you're going to copy to it.
<cbeer>and letting that give a validation error
* github-ff joins
[fcrepo4] awoods closed pull request #120: Cleanup http modules (master...CleanupHTTPModules) http://git.io/WrCrNA
* github-ff leaves
* github-ff joins
[fcrepo4] awoods deleted CleanupHTTPModules at bb068a2: http://git.io/Oa4_4w
* github-ff leaves
<ajs6f>Yeah, I'd return the error (made human-readable). People are going to use this to integrate other workflows, right?
<cbeer>sorry, that wasn't the real question, i got distracted. the existing logic will return null for "/some/invalid?bad-path"14:58
and i was trying to figure out if i should recreate that for the "just get the path" logic
but let's say the answer is no, for now.14:59
<bljenkins>Project fcrepo4 build #1219: UNSTABLE in 16 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1219/15:01
awoods: Non-functional reformatting update
<ajs6f>Even I can't believe I'm going to say this, but yet, I am: this is another place Scala wins. In Scala, you would return an Option[T], which is a monad around type T. It's either Some or None. No null pointer nonsense. Actually, Guava has an Optional<T> class, but I don't think the concept is widely used enough in Javaland to be safe for us to use.
Or are you talking about paths that are _really_ bad, like they could never exist at all.15:02
?
<cbeer>ajs6f: really bad, could never exist at all.15:04
* ksclarke leaves
<ajs6f>Ohhhh. I'd return the error, no doubt.
Maybe we should even reexamine the hbehavior of HttpGraphSubjects.15:05
<cbeer>i vote that. in phase 2 :P
<ajs6f>Phase N, the Final Frontier.
<awoods>osmandin: RE:AtomJMSIT, have you considered updating the method: waitForEntry() to actually wait for the expected entries?15:07
* ermadmix leaves
<osmandin>awoods: yeah, I changed it to 215:08
awoods: and the tests run ok, but there's this parsing exception thrown sometimes15:09
<awoods>osmandin: I mean, wait for specific entries, not just a number of entries.
osmandin: what is the parsing exception?15:10
<osmandin>awoods: it's in the ticket comment
<awoods>osmandin: What is at this line: org.fcrepo.integration.jms.observer.AtomJMSIT.testDatastreamTerm(AtomJMSIT.java:214)15:12
osmandin: It seems like your local version is different than master.
<osmandin>awoods: yes, because I add an assert for numer of entries15:13
<awoods>osmandin: yes
<osmandin>awoods: it was List<Category> categories = copyOf(entry.getCategories("xsd:string"));
<awoods>osmandin: More of the log/test-output would be helpful in the gist.15:15
* github-ff joins
[fcrepo4] cbeer created move-copy-57813074 (+3 new commits): http://git.io/2GtfOA
fcrepo4/move-copy-57813074 f9ea3f6 Chris Beer: add a method to get the path to a node from a RDF graph subject
fcrepo4/move-copy-57813074 9590d09 Chris Beer: add support for move and copy operations to the NodeService
fcrepo4/move-copy-57813074 f532cc7 Chris Beer: expose copy and move operations through the REST API
* github-ff leaves
* github-ff joins
[fcrepo4] ajs6f created ExposeRDFStream (+3 new commits): http://git.io/v6w57Q
fcrepo4/ExposeRDFStream f83b66e ajs6f: Renamed GraphSubjects to IdTranslator
fcrepo4/ExposeRDFStream 53c132d ajs6f: RdfStream can now be created from Model
fcrepo4/ExposeRDFStream 013df44 ajs6f: Now capturing the notion of repo-managed RDF predicates
* github-ff leaves
<awoods>osmandin: There are print statements that show the body of the message coming through.
<pivotal-bot_____>Andrew Woods accepted "Code cleanup for fcrepo-http-*" https://www.pivotaltracker.com/story/show/5940295415:16
Andrew Woods delivered "Run benchmarks on SCC cluster at FIZ Karlsruhe with fcrepo3 and fcrepo4" https://www.pivotaltracker.com/story/show/52289545
* tecoripa joins
<pivotal-bot_____>Andrew Woods delivered "Create wiki page with matrix of basic roles, permissions to test" https://www.pivotaltracker.com/story/show/5928733815:17
* ermadmix joins
* ajs6f leaves
* github-ff joins
[fcrepo4] cbeer force-pushed move-copy-57813074 from f532cc7 to ab2c787: http://git.io/2a-q0w
fcrepo4/move-copy-57813074 d00c2d7 Chris Beer: add a method to get the path to a node from a RDF graph subject
fcrepo4/move-copy-57813074 7e8c06e Chris Beer: add support for move and copy operations to the NodeService
fcrepo4/move-copy-57813074 ab2c787 Chris Beer: expose copy and move operations through the REST API
* github-ff leaves
<awoods>osmandin: Feel free to push your updates (if you have not already done so) to your personal github repo.15:18
<osmandin>awoods: yes, i'll do so.. i think it's working now (at least for a 150 run), not sure about the parser exception though.. I'll paste the full stacktrace here in a minute15:19
* ajs6f joins15:20
<pivotal-bot_____>Osman Din added comment: "https://gist.github.com/osmandin/7124886" https://www.pivotaltracker.com/story/show/58726512
<bljenkins>Yippie, build fixed!15:22
Project fcrepo4 build #1220: FIXED in 21 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1220/
Project fcrepo-fixity-corrupter build #295: SUCCESS in 1 min 26 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/295/15:24
<pivotal-bot_____>Chris Beer added comment: "https://github.com/futures/fcrepo4/pull/123" https://www.pivotaltracker.com/story/show/57813074
* github-ff joins
[fcrepo4] cbeer opened pull request #123: Add support for moving and copying nodes via the java and REST APIs (master...move-copy-57813074) http://git.io/hMUCtg
* github-ff leaves
<cbeer>ajs6f: yes, i think the graph subjects are due for a refactor.. and it probably should include the subject injection mechanics too15:25
<ajs6f>+1 to that.15:26
<awoods>cbeer: Are you using that PR as an in-progress workspace, or it is ready to go?
<ajs6f>I'd also like to change the name of GraphSubjects to IdTranslator.
<cbeer>awoods: ready to go if you think so.
or, at least, open to nitpicking
<awoods>cbeer: Is there an associated ticket?
<ajs6f>Damn, he's fast.
<cbeer>awoods: in the body of the PR
ajs6f: fast, if it's any good.
<ajs6f>I'm not worried.15:27
<cbeer>webdav took away most of the thinking, at least
<ajs6f>That's a good quote.
You could take it a number of ways.
<cbeer>i suspect we'll want to add a non-webdav-verb endpoint for lazy people eventually15:28
because testing it with httpclient was annoying
<ajs6f>I just want to avoid making any more fcrepo-specific API than we really have to.
No gridFTP!
afk bbs
* ajs6f leaves
* ksclarke joins15:30
* ajs6f joins15:31
barmintor: My wife just emailed me that she's landed in Austin (I'm going to meet her there in a few days). First order of business: a trip to the nearest Taco Cabana. {grin}
<awoods>cbeer: I will take a closer look in a bit.15:33
* kaarefc leaves
<pivotal-bot_____>Andrew Woods finished "Implement node/tree copy." https://www.pivotaltracker.com/story/show/5781307415:34
Osman Din finished "Create fcrepo4 multi node benchmarking result on the Yale cluster" https://www.pivotaltracker.com/story/show/5925305015:36
<barmintor>ajs6f: oh my
ajs6f: Can I make a rec?15:37
<ajs6f>barmintor: I could watch those tortilla making machines for hours.
barmintor: Please!
<barmintor>ajs6f: Go to a place called Curra's on Oltorf
<ajs6f>barmintor: I'll pass it along pronto. She's an Austin native too, and she's got a long list of foods to put herself outside of…15:38
<bljenkins>Project fcrepo-fixity-corrupter build #296: SUCCESS in 1 min 37 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/296/15:46
Project fcrepo-kitchen-sink build #547: STILL UNSTABLE in 4 min 51 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/547/15:51
* github-ff joins16:05
[fcrepo4] ajs6f pushed 1 new commit to ExposeRDFStream: http://git.io/1qKNrA
fcrepo4/ExposeRDFStream 31380cb ajs6f: Providing RDF stream that distinguishes managed and unmanaged triples
* github-ff leaves
<ajs6f>Cripes!16:09
cbeer: Were we translating between rdf:type assertions and JCR mixins? Looks like we were...
<cbeer>ajs6f: yep.16:10
<ajs6f>Urg. I completely missed that.
k
I'll turn around and fix that before going on.
New ticket.
<pivotal-bot_____>A. "Senphitic" Soroka added "Replace functionality to translate between rdf:type and JCR mixins" https://www.pivotaltracker.com/story/show/5944113016:11
A. "Senphitic" Soroka started "Replace functionality to translate between rdf:type and JCR mixins" https://www.pivotaltracker.com/story/show/59441130
<ajs6f>cbeer: I made you requester, just so I can get you to look at what all I've been doing with this RDF iteration stuff.16:12
* github-ff joins16:13
[fcrepo4] ajs6f created FixRDFTypeJCRMixinTranslation from master (+0 new commits): http://git.io/eVSs3Q
* github-ff leaves
<cbeer>ajs6f: sanity check. i'm trying to pull together some sort of uber-class that we can use to translate graph uris (including REST API uris) to jcr sessions and paths, and vice versa.. here's the interface i've landed on so far (and i'm not thrilled with it..) https://gist.github.com/cbeer/16d9e7d138b8ddf0f55f16:20
<ajs6f>Mm… I think we have two different contracts here.16:21
<cbeer>agreed. i'm not sure how separable they are, though.16:22
<ajs6f>Fair enough.
How come theres a switchable Session?
<bljenkins>Project fcrepo4 build #1222: UNSTABLE in 17 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1222/
<ajs6f>When would you want to swap Sessions midway through?
<cbeer>i'm not sure there's a real need for it. i was thinking something to do with workspaces or transactions16:23
<ajs6f>Mm… could be. But I'm not sure we should start with it.
<cbeer>works for me16:24
anyway, totally crazy? should i work on trying to disentangle the contracts where they are tangled now?
or just run with this nasty thing
<ajs6f>Not crazy, but I think we can...
I'm trying to think about it in terms of self-bounded generics.
* kaarefc joins16:25
<ajs6f>Like we start with a "Builder" class (thats the self-bounded one)
and add two subclasses (one for each contract)...16:26
er, interfaces...
That would do it, I think.
Does that make sense? Should I gist something?16:28
<cbeer>ajs6f: i think so.. the two contracts are the path + session stuff, vs the resource stuff?16:34
<ajs6f>That's how I'm thinking about it.16:35
<pivotal-bot_____>Osman Din added comment: "Patch for testAtomStreamNodePath stacktrace: ""
https://github.com/osmandin/fcrepo4/commit/f6c836c7b430f2e6f523f1..." https://www.pivotaltracker.com/story/show/58726512
<cbeer>ajs6f: makes sense.. and then we can tack on a third contract for http uris, i think
<ajs6f>For now, we don't associate the mapping between internal and external ids to a given Session.
cbeer: would the http stuff be in one of the impls of this stuff
?
<cbeer>in -kernel, yes.. in -http, external ids can include some session details
<ajs6f>Oohhhh. I didn't know that.16:36
<cbeer>yeah, we smuggle workspace and transaction info on the external id
<ajs6f>Oh, right, right...
Make sense.
Worlds within worlds...
What you see depends on what kind of eyes you use
<cbeer>so, yeah, http would be an impl of this, and probably adds a couple new methods (to get the servlet context, i guess)16:37
<ajs6f>Sure. That's the right place for that stuff. I really want to change names, too. GraphSubjects is totally confusing. It confused awoods a week or two ago.16:39
I know why barmintor picked it, but we should change it now.
<cbeer>IdentifierBuilder?16:40
<ajs6f>Sounds _much_ better to me.
+1
<barmintor>Boo.16:41
<ajs6f>barmintor: Tell you what, to make up for it, you can buy me a beer.16:42
<barmintor>HOW MANY BEERS CAN I PUT YOU DOWN FOR?
<ajs6f>As many as I can put down. {grin}
Empty.
<bljenkins>Yippie, build fixed!16:44
Project fcrepo4 build #1223: FIXED in 21 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1223/
Project fcrepo-fixity-corrupter build #297: SUCCESS in 1 min 39 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/297/16:46
<ajs6f>Headin' out. See y'all tomorrow.
* ajs6f leaves16:47
<barmintor>adios
* osmandin leaves16:49
<bljenkins>Project fcrepo-kitchen-sink build #548: STILL UNSTABLE in 3 min 18 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/548/
* kaarefc leaves17:08
* gregjansen leaves17:34
* ermadmix leaves17:56
* github-ff joins18:15
[fcrepo4] cbeer created identifier-builder (+1 new commit): http://git.io/s3hAHQ
fcrepo4/identifier-builder e66a189 Chris Beer: working on refactoring a identifier builder that can translate from some combination of session, node, rdf resource and http uri.
* github-ff leaves
* ksclarke leaves18:21
* ksclarke joins18:29
<bljenkins>Project fcrepo4 build #1224: UNSTABLE in 15 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1224/18:30
* tecoripa leaves
* osmandin joins19:08
awoods: regarding AtomJSIT I caught another one19:12
<pivotal-bot_____>Osman Din added comment: "Another org.apache.axiom (exception caught in generic try/catch block in onMessage() - not sure about the des..." https://www.pivotaltracker.com/story/show/5872651219:15
* fasseg leaves19:20
* osmandin leaves19:23
<pivotal-bot_____>Andrew Woods added comment: "It looks like the SolrIndexerIT.java integration test already exists. Is there anything else that needs to ..." https://www.pivotaltracker.com/story/show/5745328020:33
Andrew Woods added comment: "In order not to spend too much time on this integration test issue, maybe try pulling testDatastreamTerm() ..." https://www.pivotaltracker.com/story/show/5872651220:40
Andrew Woods accepted "Run benchmarks on SCC cluster at FIZ Karlsruhe with fcrepo3 and fcrepo4" https://www.pivotaltracker.com/story/show/5228954521:04
Andrew Woods accepted "Create wiki page with matrix of basic roles, permissions to test" https://www.pivotaltracker.com/story/show/5928733821:20
* ksclarke leaves21:21
<pivotal-bot_____>Andrew Woods accepted "Create fcrepo4 multi node benchmarking result on the Yale cluster" https://www.pivotaltracker.com/story/show/5925305021:22
Andrew Woods edited "Research and design the REST semantics for object (and tree) copying" https://www.pivotaltracker.com/story/show/5781296021:23
* ksclarke joins
<cbeer>awoods: ping?21:28
<awoods>cbeer: hi21:29
<cbeer>awoods: i've been trying to update some of our dependencies and have run into some trouble with activemq
i'm not sure if they've done a better job segregating dependencies or what
<awoods>cbeer: need a hand?
<cbeer>activemq-stomp and activemq-kahadb-store aren't in the dependency tree any more from fcrepo-jms21:30
and are only used by fcrepo-webapp, seemingly
<awoods>seems reasonable to pull them directly into fcrepo-webapp, right?
<cbeer>do you think i should wire those dependencies into webapp?
yeah
<awoods>sure
<cbeer>although i also don't know why -webapp needs them
other than maybe by default21:31
<awoods>hmm
Where does the dependency error show up?
<cbeer>running the fcrepo-webapp ITs
<awoods>compile time?
got it
do you know which one?
<cbeer>sanity, maybe. it's the one that loads the jms.xml spring config21:32
<awoods>if necessary, at least you should be able to include those dependencies with <scope>test</scope>
<cbeer>i think we wire in activemq by default when you run it too21:33
<awoods>it does not seem like it should be necessary, does it.21:34
<cbeer>maybe it was added for the islandora jms-based indexing?
does seem weird to include it by default though21:35
<awoods>It seems like all of the jms dependencies should be inherited from fcrepo-jms
<cbeer>except these are dependent on your broker config21:36
i guess pulling them into fcrepo-jms is apparently no worse than it was before21:37
<awoods>you mean, fcrepo-webapp/atom_jms.xml is dependent on fcrepo-jms/jms.xml ?21:38
or what exactly depends on the broker config?21:39
<cbeer>awoods: sorry. i'll start again.
in fcrepo-webapp, we have a activemq broker config
that requires activemq-stomp and activemq-kahadb-store
i guess those used to be somewhere in the dependency tree from fcrepo-jms.. or weren't broken out yet
a) why does fcrepo-webapp need a broker config that uses those21:40
b) if we want them, we can either wire them into -jms, as they were before or into -webapp, which is actually using them
<awoods>There has been the long-standing discussion of having the option of removing fcrepo-jms out of the main codebase...21:41
<cbeer>meaning not wiring it into -webapp at all?21:42
<awoods>The question is, "how do we wire it into webapp, flexibly"?
It would be nice to have all of the jms junk in the fcrepo-jms module21:43
<cbeer>back after dinner, and i'll do something.21:44
<awoods>p.s. As a side note, we should probably populate the openwire and stomp ports using the maven plugin: build-helper.
* fcrepo-bot joins21:55
<pivotal-bot_____>Andrew Woods added comment: "Looks good, pending code-review comments." https://www.pivotaltracker.com/story/show/5781307422:10
<awoods>cbeer: talk to you in the morning.22:15
bbitm
* github-ff joins
[fcrepo4] cbeer created dep-update (+1 new commit): http://git.io/b4EArQ
fcrepo4/dep-update df46dce Chris Beer: bump upstream dependencies to latest versions
* github-ff leaves
* fcrepo-bot leaves22:58