Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* jcoyne leaves00:09
* 7JTAAF21Y joins06:43
* mohamedar joins07:14
* mohamedar leaves07:15
* diegopino joins07:30
* jcoyne1 joins07:58
* diegopino_ joins08:00
* diegopino leaves08:03
* jcoyne1 leaves08:05
* dhlamb_ joins08:07
* 7JTAAF21Y leaves08:16
* jrgriffiniii joins08:55
* mohamedar joins09:01
* bseeger joins09:12
* whikloj joins09:21
* acoburn joins09:31
* aelkiss leaves09:45
* bseeger leaves10:00
* bseeger joins10:16
<diegopino>acoburn: hi, when talking about non-modeshape options for f4 in the future, those that mean also non infinispan?10:20
<acoburn>diegopino: yes
<diegopino>acoburn: mmm...
acoburn: sounds nice
<acoburn>diegopino: in fact, MODE 5 will not use ISPN10:21
<diegopino>acoburn: i just saw i pull allowing leveldb swap for MYSQL. Is that at infinispan level right?10:22
<acoburn>diegopino: yes that is at the infinispan level10:23
* osmandin joins
<diegopino>acoburn: ok, this sounds fine. So if getting rid of infinispan is an option in the future, some crazy ideas like playing with rockdb… are too crazy? =)10:24
acoburn: i have seen that the cachestore options for infinispan are quite ample10:25
<acoburn>diegopino: I'm not familiar with RockDB.10:26
<diegopino>acoburn: but i was reading today some stuff on rockdb and found it very very interesting for many applications.
<acoburn>diegopino: looks like it's c++
<diegopino>Yeah, but i has some Java dings there….https://github.com/facebook/rocksdb/wiki/RocksJava-Basics10:27
acoburn: also very specific to SSD, but it's pretty fast10:28
* jcoyne joins10:29
<diegopino>acoburn: this is their benchmark, https://github.com/facebook/rocksdb/wiki/Performance-Benchmarks10:30
acoburn: what i have noticed is how leveldb perfomance gets slower and slower when the db does not fit in memory10:31
acoburn: ok, i'm just experimenting. Thanks
<acoburn>diegopino: experiment away10:32
<awoods>diegopino: As a simple starting point, it would be great if you could do some comparative testing with the MySQL backend.10:33
<diegopino>awoods: Hi, yes i will. I really don't expect much from Mysql write speed. But i know that under certain confs, read is a lot faster than leveldb10:34
awoods: and then again i would fail there because i don't know how big my fedora4 must be to have some comparable results.10:35
<awoods>diegopino: it would be interesting to see if no such slowdown is experienced at the scale where leveldb begins to deteriorate.
* peichman joins10:36
<diegopino>awoods: That should be easy to test, because i almost have no RAM =)
awoods: will find some time during the next weeks to test.
<diegopino>awoods++ acoburn++
* whikloj leaves10:43
* dhlamb_ leaves10:47
* ajs6f joins
* mcritchlow joins10:48
* bseeger leaves11:00
* bseeger joins11:04
* mikeAtUVa joins11:05
* diegopino leaves11:25
* mcritchlow leaves11:43
* bseeger leaves11:52
* acoburn leaves11:53
* dhlamb joins12:05
* dhlamb leaves12:09
* dhlamb joins12:17
* whikloj joins12:58
* jcoyne1 joins12:59
* jcoyne leaves13:00
* mcritchlow joins13:02
* awoods leaves
* awoods joins13:06
* peichman leaves
<whikloj>mikeAtUVa: versioning question if you have a second?
* diegopino joins13:09
* bseeger joins13:10
<mikeAtUVa>whikloj: I have a second, but probably no insight.... ask away.13:16
<whikloj>mikeAtUVa: If I have a FedoraResource (ie. /versionResource), then I create a version, I get it with the jcr:storage path. Can I get the "normal" path from the version, or do I need to got to the baseVersion?13:18
sorry "I get the version with the jcr...."
I've looked at the JCR spec, but some of it is over my head.13:19
* acoburn joins
<mikeAtUVa>whikloj: let's see....
The spec says that the versions are always to be stored in /jcr:system/jcr:versionStorage "If a repository supports full versioning, then it must expose the version storage at /jcr:system/jcr:versionStorage. If it supports only simple versioning then it may expose the version storage (see §3.13.8 Version Storage)." http://www.day.com/specs/jcr/2.0/3_Repository_Model.html#
<whikloj>so the path of the version is always in the system space
<mikeAtUVa>whikloj: this poses some problems for us in that: 1) we can't really "view" versions in any reasonably easy way.
whikloj: yeah... if we continue to use the jcr versioning for our versions.13:22
<acoburn>whikloj: mikeAtUVa: the JCR spec may say that, but Fedora is *not* implementing the JCR specification13:23
<mikeAtUVa>whikloj: JCR kind of limits us to "creating" versions and "restoring" versions...
<whikloj>acoburn: go on...
<acoburn>whikloj: Fedora is built on top of Modeshape, which implements JCR, but those JCR notions are not part of any (yet to be written) Fedora specification13:24
<mikeAtUVa>acoburn: I'm aware of that... in which case if our requirements don't align with JCR, we'll have to reimplement a versioning system that meets our requirements. Which has been a problem from the beginning since we don't really have requirements.
<acoburn>mikeAtUVA: isn't that where memento comes in?
<whikloj>mikeAtUVa: thanks for the help13:26
mikeAtUVa, acoburn: I hear what you are saying and I agree that memento will take over. But for now we are using a very Modeshape specific versioning.13:28
mikeAtUVa, acoburn: I'm guessing when we start looking to implement memento we should keep things like WebAC accessTo controls in mind13:29
<mikeAtUVa>acoburn: does mimemto have an api for creating versions?13:33
whiklojo, acoburn: my gut feeling is we're going to have to not use JCR versioning, (or at least do some nasty stuff behind the scenes) in order to support JCR. Specifically, we need to be able to expose historic versions in a reasonable way... which whiklojo has discovered we cannot.13:34
<whikloj>mikeAtUVa, acoburn: is memento just a way to expose historical versions?13:35
<mikeAtUVa>It might be worth a little investigation to see if we could easily replace our code that "views" old versions (that secretly and problematically exposes stuff in the system space) with code that instead copies the versioned node somewhere and reverts that copy to the version the user requests. That copied node would behave more like our other resources.13:37
<acoburn>whikloj: memento is primarily a way to access (read) versions, but there have been conversations about what a write interface would be
<mikeAtUVa>That approach would make our Modeshape Fedora implementation potentially weak on version viewing, but strong on version creation.13:38
<whikloj>acoburn: ok cool, I was not aware of that.
mikeAtUVa: the memento approach?
<mikeAtUVa>whikloj, no the system-space workaround I described in my previous comment.13:39
<whikloj>mikeAtUVa: ahhh, I guess my only concern would be what is the lifecycle of that copied resource.13:40
* dhlamb_ joins
<mikeAtUVa>whikloj: yeah, there's some messy concerns... but as it is when we show a user a rendering of a jcr:frozenNode and say it's their old version, we're exposing JCR stuff and presenting an inaccurate representation of that version.13:41
whikloj: we don't properly fake all the stuff in the frozenNode to make it look like the version actually looked... we *could* and probably *should*, but the first lazy implementation did not.13:42
* mikeAtUVa takes full blame for the first lazy implementation.
* dhlamb leaves
<whikloj>mikeAtUVa: I'm weak on this whole versioning thing, so what is missing/inaccurate from our current representation?13:45
* bseeger leaves13:53
* mcritchlow_ joins14:02
* mcritchlow leaves14:04
* dhlamb_ leaves14:14
* dhlamb joins14:15
* diegopino leaves14:16
* mcritchlow joins14:34
* mcritchlow_ leaves14:36
* jcoyne1 leaves14:43
* jcoyne joins14:45
* bseeger joins14:52
<ajs6f>mikeAtUVa: awoods is pursuing a conversation with the Memento crew about the write question. I don't know how far it has gotten, but he remarked a few weeks ago that they've been very receptive to the idea and see it as a natural extension of the previous work.14:57
<awoods>ajs6f: true, although I will note that there is fairly broad familiarity with the "create-a-version-on-every-update" model. That is the model that memento is currently founded on.15:00
* mcritchlow leaves15:01
<ajs6f>awoods: Right on, brother. The work, then, is just to formalize that (for the Memento squadron) and for us, to factor Memento and LDP together to create a kind of shark-octopus supermonster of versioning model excellence.
* dhlamb leaves15:02
* bseeger leaves15:06
<whikloj>ajs6f, awoods: Is that the direction we are thinking of going? Create a version on every update, ala Fedora 3?15:11
<ajs6f>whikloj: I think we want to maintain the choice people have, of autoversion or not.15:12
<awoods>ajs6f: we have removed the "autoversion" choice
<whikloj>ajs6f: Fedora 4 doesn't have autoversion
* mcritchlow joins15:13
<whikloj>ajs6f, awoods: not that I am necessarily opposed to having an autoversion option
<ajs6f>awoods: I mean autoversion as in, create a new version every time I post to a resource for which I have slected that feature. You are say we don't offer that now? So I have to send some kind of special header to create a version. EVERY time.
<awoods>whikloj: the on-demand approach we currently have is simple, we will want to determine what users actually want during the specification discussions.15:14
ajs6f: yes
<ajs6f>awoods: So I can have a resource that I want to version, and I send updates to it, and if I accidentally forget that header, I overwrite what's there. Permanently.15:15
<whikloj>ajs6f: You have to POST to <resource URI>/fcr:versions to create a version (I believe)15:16
<ajs6f>whikloj:awoods: We can do better than magic URIs. We need to understand how versioning implies subtypes of the basic LDP types and how the interaction models for those subtypes appear and in what way they relate to the basic ixn models.15:17
whikloj:awoods: In fact, I think we have a ticket for that.
<awoods>ajs6f: fyi: https://wiki.duraspace.org/display/FEDORA4x/RESTful+HTTP+API+-+Versioning15:18
<whikloj>awoods: We currently make use of Fc3 versioning on our main "datastream" (the Archival copy) and metadata, but not on any derivatives we store. That gives us a happy balance between data security and storage requirements.15:20
ajs6f: I don't think I understand how you would use sub-types of LDP types for versions, but I agree that there needs to be a better way.15:23
<ajs6f>whikloj: When you create a new LDP resource, you necessarily select a type. E.g. Basic Container, NonRdf Source, etc. Versioning is a "mixin" for these, e.g. "Versioned Basic Container", etc.15:24
<whikloj>ajs6f: ok, I see what you mean15:25
ajs6f: currently we seem to add the type "nt:version"
ajs6f: which IMHO makes it very tightly bound to the JCR15:26
<ajs6f>whikloj: No, that's a slightly different use of the word type. The kind of type you are thinking of is the kind of thing that can be the object of a triple.
whikloj: The kind of type to which I am referring is what determines what interaction model you are going to get.
whikloj: For convenience, we do publish that second kind of type as one of the first kinds, but they are distinct.15:27
<whikloj>ajs6f: ok, but I don't think we currently have any LDP sub-types for versioned resource. correct?
<ajs6f>whikloj: Right, because they don't exist.. We would (I think we _do_ need) to invent them and we must do that in concert with the LDP Next group.
<whikloj>ajs6f: currently I don't think we have any differences between the original (say FedoraResource/Container) and the version (which seems to still be a FedoraResource/Container). So we would (in concert with LDPNext) create a altered class with some additional methods specific to being a version15:30
or am I still in the weeds15:31
<ajs6f>whikloj: No, it's not that kind of type/class either. {grin} Check this out:https://www.w3.org/TR/ldp/#fig-ldpc-types We're interested in extending _that_.15:32
* bseeger joins15:33
* whikloj is going avoid versioning and just have a drink instead15:35
<ajs6f>whikloj: Just think about versioning in terms of scotch. You have your young stuff, then you have your 12 year, 15 year, etc.15:36
* whikloj is now checking out some 10 year old version15:37
<ajs6f>acoburn: Did you just run out o time for https://jira.duraspace.org/browse/FCREPO-1672 or did somthign stop you?15:55
<acoburn>ajs6f: actually, I thought escowles had taken it on, but my name was still on it.15:56
<ajs6f>acoburn: Oh, cool.
acoburn: Just as long as someone is kicking the ball down the field.
<acoburn>ajs6f: I'd really like to see it move forward, too15:57
ajs6f: right now I'm still plugging away on fcrepo-1312
<ajs6f>acoburn: https://www.youtube.com/watch?v=uyaJ8eR9tzw
<acoburn>ajs6f: "always twirling, twirling toward freedom"15:58
<ajs6f>Fedora 4: Twirling, twirling, twirling.
<whikloj>I love that episode16:00
* mcritchlow leaves
<ajs6f>I think of awoods and dwilcox as Kang and Kodos.16:01
* github-ff joins16:02
[fcrepo-transform] awoods opened pull request #7: Ensure transformed dates are serialized as formatted text (master...fcrepo-1877) https://git.io/vggYR
* github-ff leaves
* mcritchlow joins16:04
<whikloj>ajs6f: https://www.youtube.com/watch?v=M0aNxzF7MAk
<ajs6f>whikloj: That pretty much is the current election.16:09
<whikloj>ajs6f: Come on! You guys are having a once in a lifetime election. This could be Trump vs Sanders!!!16:10
<ajs6f>whikloj: We are having a clear and straightforward demonstration of why parlimentary systems are superior to presidential systems.16:11
* jcoyne gets up on the desk and boos
<ajs6f>jcoyne: That means about as much as your vote.
<whikloj>ajs6f: Meh, don't look to us north of 49. Our first past the post system is pushing us to be two parties.16:12
can I cast a jena:Node into either a jcr:Node or a fedoraResource?16:14
<ajs6f>whikloj: No, you want to use a translator for that. a jena node is an RDF node.
<whikloj>ajs6f: ok like a PrefixingIdentifierTranslator to change the info:fedora/blah to something usable16:17
<ajs6f>whikoj: Ish. You want an IdentifierConverter<Resource, FedoraResource>. Take a look at the API classes in fcrepo-http-api16:18
<whikloj>ajs6f: sorry it would be a DefaultIdentifierTranslator, and I have one of those :)
<ajs6f>whikloj: yeah, DefaultIdentifierTranslator < IdentifierConverter<Resource, FedoraResource>
whikloj: But where are you working? Sometimes you want to accept the translator as a part of the context.16:19
<whikloj>ajs6f++ # thanks, converting stuff from JCR <-> Jena <-> Fedora always confuses me
ajs6f: WebAC
<ajs6f>whikloj: Are you working in an HTTP API class, or what?
<whikloj>ajs6f: I'm the WebAC authorization delegate, https://github.com/fcrepo4/fcrepo-module-auth-webac/blob/master/src/main/java/org/fcrepo/auth/webac/WebACRolesProvider.java16:21
ajs6f: or part of the authorization delegate, I'm never quite sure.
<ajs6f>whikloj: yeah, it doesn't look like you have a contextual translation, but one last question: where di you get the Jena Node from?16:23
* peichman joins16:24
<whikloj>ajs6f: I got the triples off of a version of a resource using FedoraResource.getTriples()
<jcoyne>ajs6f: parlamentary systems do sound great: http://www.telegraph.co.uk/news/uknews/crime/9100013/Eric-Joyce-MP-arrested-fears-over-Parliaments-drinking-culture.html
* jrgriffiniii leaves
<ajs6f>whikloj: Okay, then your best bet is probably DefaultIdentifierTranslator, as you thought. Sorry to have slowed you down16:25
<whikloj>ajs6f: it's okay, DefaultIdentifier wants a Resource, I have a Node. am I missing a step?16:26
<ajs6f>jcoyne: Thank goodness our Congress is composed entirely of morally upstanding and rigorously respectable people. Seriously, let's leave it. It's not interesting to debate something over which we have no power.
<whikloj>jcoyne: you don't want parliamentary, you want something that is actually going to get what people want.16:27
<jcoyne>I didn't really want a debate. I just want someone to pay for my bar tab.
<whikloj>ajs6f, jcoyne: doesn't the fact that you feel you have no power, seem the opposite of a democracy. You need to make voting mandatory!!! Like Australia.16:28
<ajs6f>whikloj: No, you're cool. Just call ::asNode() on your Resource.16:29
whikloj: Oh, wait, sorry, you're going the other way.
<whikloj>ajs6f: yeah
jena node -> Resource
<ajs6f>whikloj: If you have a Jena Model in hand use that to translate.16:30
whikloj: Do you?
<whikloj>ajs6f: I don't currently, I could make a defaultModel
<ajs6f>whikloj: You got the Node from a Triple, right?
<whikloj>ajs6f: Yes
<ajs6f>whikloj: Which you got from a RDFStream, right?
<whikloj>ajs6f: yes..I think I get where you are going.16:32
ajs6f: just not sure where the end result is
<ajs6f>whikloj: myRdfStream.toModel()
whikloj: Then get your Resource directly from thaht.16:33
whikloj: Or are you needing to operate in a pure-streaming mode? Is this something that is going to be called on every request?
<whikloj>ajs6f: It will be called on any request when using WebAC and the resource being acted on is a version16:34
<ajs6f>whikloj: Erm. That's a little sensitive. I'm starting to think we need more fine-grained machinery than getTriples. But that doesn't help you here and now.16:36
<whikloj>ajs6f: so would I be doing RdfStream.asModel().getResource(String uri)
ajs6f: If I can get this working, then I can at least open a PR and you all can look and see if what I am doing is any good to begin with16:37
<ajs6f>whikloj: Y'yeah. Do that for now, because this isn't the point of what you're doing anyway, and we'll fix it in the mix.
acoburn: Do you see what I mean about getTriples? But on the other hand, we don't ever want to assume on random access to properties in the API...16:38
whikloj: Actually, I'd have to do look it up, but I think you want RdfStream.asModel()::getProperty, but you can play with it and see.16:39
<whikloj>ajs6f: ok I'll check that out16:40
* mcritchlow leaves
<ajs6f>whikloj: For future reference: http://answers.semanticweb.com/questions/14287/functions-to-convert-between-jena-nodetriple-and-rdfnodestatement/14291 That's Andy Seaborne, so you know it's good info.16:42
<whikloj>ajs6f: thanks16:43
<ajs6f>awoods:acoburn: Using Resource for translation is wrong. We should be using Node. I didn't realize we were doing that at the time, but it's such a lot of pointless work that we should probably clean it up.16:45
<acoburn>ajs6f: you mean jena's Node (not jcr.Node), right?16:47
<ajs6f>acoburn: Right, right.
* osmandin leaves
<acoburn>ajs6f: yes, that make sense
<ajs6f>acoburn: Resource implies a context that almost never actually exists. The code smell is seeing ModelFactory.createDefaultMode() called _just_ to translate Nodes into Resources.
<acoburn>ajs6f: yes, there's a lot of that in the code16:48
<ajs6f>acoburn: There's a lot of it about.
acoburn:awoods: https://jira.duraspace.org/browse/FCREPO-190016:50
* ajs6f leaves17:00
* mikeAtUVa leaves17:04
* jcoyne leaves17:07
* whikloj leaves17:14
* peichman leaves
* jgpawletko joins17:32
* jgpawletko leaves17:36
* bseeger leaves17:45
* esm_ joins18:02
awoods: I've squashed my PR for FCREPO-1875. Do I just re-push the branch and overwrite the the history, or do you prefer that another PR be opened? Just wanted to make sure of the process.18:03
<awoods>esm_: feel free to perform a --force push to the existing branch/PR18:04
<esm_>awoods: copy!18:05
* acoburn leaves18:08
* esm_ leaves18:09
* mcritchlow joins18:25
* mcritchlow leaves18:26
* diegopino joins19:12
awoods: hi, i know its late19:13
awoods: do you know if the text/turtle parser in F4 (for example in API POST) is strict?19:14
awoods: maybe i'm just getting confused, but i see at the API doc that some rdf passed to the body as text/turtle don't end with a dot (.)19:15
awoods: confused also because it's late...
<awoods>diegopino: I believe the text/turtle parser requires valid turtle... but it would be interesting to see otherwise.19:16
<diegopino>awoods: thanks. Well the specs say each tripple should end with a dot. Example does not and works fine. my library is more strict and fails.19:19
awoods: is not that bad, it's just our microservices, which add/modify some tripples before passing to f4 will reject those not strict serializations.19:21
<awoods>diegopino: hmm... that sounds like a bug. Would you mind putting in a ticket with reference to: https://www.w3.org/TR/turtle/#sec-grammar19:22
<diegopino>awoods: Ok, will do first in the morning. My brain is shutting down. Thanks!19:23
* mohamedar leaves
<awoods>diegopino: thanks... and stay rested.
* mohamedar joins
<diegopino>awoods: oh… REST…no19:24
<diegopino>awoods++ see you
* diegopino leaves
* mohamedar leaves19:25
* jcoyne joins19:27
* mohamedar joins19:31
* mohamedar leaves
* mcritchlow joins
* jcoyne leaves19:36
* github-ff joins19:38
[fcrepo-audit] awoods opened pull request #28: Remove method: FedoraResource.setURIProperty() (master...fcrepo-1894) https://git.io/vggj9
* github-ff leaves
* jcoyne joins19:44
* dhlamb joins19:45
* mcritchlow leaves19:53
* jcoyne leaves19:54
* jcoyne joins19:57
* jcoyne leaves20:02
* dhlamb leaves20:31
* dhlamb joins20:58
* jcoyne joins21:00
* dhlamb leaves21:31
* jcoyne leaves22:06
* jcoyne joins23:19
* jcoyne leaves23:35