Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* mcritchlow joins01:19
* mcritchlow leaves01:23
* f4jenkins leaves03:52
* f4jenkins joins
* awoods leaves04:18
* awoods joins04:52
* dwilcox joins07:45
* jcoyne leaves08:02
* jcoyne joins08:03
* jcoyne leaves08:06
* mohamedar joins08:14
* osmandin leaves08:21
* osmandin joins08:22
* esm_ joins08:27
awoods: morning! just acking your ping re testing FCREPO-1877. I hope to get to that today.
* dhlamb joins08:43
* mohamedar leaves08:51
* jgpawletko joins
* whikloj joins09:03
* mikeAtUVa joins09:13
<awoods>esm_: thanks09:31
* github-ff joins09:41
[fcrepo-camel-toolbox] awoods pushed 1 new commit to master: https://git.io/vgKnE
fcrepo-camel-toolbox/master 8523ac9 Aaron Coburn: Move features.xml provisioning into its own module...
* github-ff leaves
* github-ff joins
[fcrepo-camel-toolbox] awoods closed pull request #71: Added PAX Exam to testing framework (master...fcrepo-1897) https://git.io/vg0EU
* github-ff leaves
* bseeger joins09:42
* ajs6f joins09:44
* github-ff joins09:51
[fcrepo-java-client] awoods pushed 4 new commits to master: https://git.io/vgKCN
fcrepo-java-client/master d4197d1 Elliot Metsger: FCREPO-1875: Implement Closeable for FcrepoResponse.
fcrepo-java-client/master b8ccaaa Elliot Metsger: FCREPO-1875: Free the HTTP connection by closing the response body when an FcrepoOperationFailedException will be thrown.
fcrepo-java-client/master 6031791 Elliot Metsger: FCREPO-1875: Use CloseableHttpResponse and CloseableHttpResponse.close() to release HTTP response resources instead of manually closing the underlying HttpResponse InputStream.
* github-ff leaves
* mohamedar joins09:52
* osmandin leaves
* github-ff joins
[fcrepo-java-client] awoods closed pull request #7: Addresses FCREPO-1875 (master...FCREPO-1875) https://git.io/vzoYd
* github-ff leaves
* travis-ci joins
fcrepo4-exts/fcrepo-java-client#17 (master - de59b95 : Elliot Metsger): The build passed.
Change view : https://github.com/fcrepo4-exts/fcrepo-java-client/compare/912444a4873c...de59b95e1de6
Build details : https://travis-ci.org/fcrepo4-exts/fcrepo-java-client/builds/108552020
* travis-ci leaves
* esm_ leaves10:16
<ruebot>awoods: i'm in meetings most of today (gonna miss the tech call), but I'll try and test jmeter pull later today time permitting.10:21
<awoods>ruebot: thanks for the heads-up and jmeter help.10:22
* mcritchlow joins10:24
* jcoyne joins10:25
* peichman joins10:28
* diegopino joins10:32
* mcritchlow leaves10:58
* kat3_drx joins10:59
* diegopino leaves11:00
* yinlin joins11:03
* ajs6f is here.
<ajwagner>Just got out of my last meeting and calling in now.
* tjohnson joins11:04
<ajs6f>awoods: How would that work with something like federation, when the persistence can change under the repo without the repo knowing.11:09
* mcritchlow joins11:13
<ajs6f>Specs are nice.11:19
* ksclarke joins11:20
<ajs6f>awoods: You are breaking up badly.
* jrgriffiniii joins
<ajs6f>Ooh, that's nice music.11:22
I think Wendy Carlos did a version of this.
Can we have this for our hold music?11:23
You contextualized the heck out it.11:27
<ajwagner>My only comment is: It's great to see this happening ;)
<jrgriffiniii>Sorry, emergency issue, must leave immediately11:29
* ksclarke_ joins11:31
* Guest14147 leaves
<ajs6f>barmintor: That is an amazing guitar pedal you are using.11:36
<barmintor>trachea by Vox
* anarchivist leaves11:37
* anarchivist joins11:38
<ajs6f>https://gitter.im is what I was thinking of.11:42
Stop talking all at once!11:43
<yinlin>google doc + 111:47
<bseeger>google docs ++
<whikloj>google docs is good +1 # Make sure to only allow people to suggest11:48
<ajs6f>Let's GOOOOOO!11:50
whikloj: Do you know if a Google Doc can be set to disallow editing mode?
<escowles>ajs6f: you can allow only some people (e.g., only the creator) to edit, and only let other people suggest or comment
<ajs6f>escowles:whikloj: Cool, we need to remember that. For barmintor's sake. ;)
<barmintor>awoods: so do we have a folder we can put the stub documents in?11:52
<whikloj>awoods: I'll try at batch/atomicity
let bseeger volunteer acoburn
ajs6f: escowles: You can only restrict shared Google docs to edit, comment, or view. No suggesting11:55
<ajs6f>whikloj: Foo. Well, you can still suggest if you remmeber to turn it on, right?11:56
<whikloj>ajs6f: I am guessing if you have edit privileges, then yes
<bseeger>I'm guessing acoburn will be fine with it, but I can't speak for him. I'll be happy to fill in if he's unable to. Though honestly I don't know much about this (yet).11:57
<ajs6f>whikloj: That ought to be enough, given courteous use.
<whikloj>ajs6f: Ohh actually just tested and comment mode is the same as suggesting
<ajs6f>bseeger: That's the only prerequisite for work on Fedora. Anyone who knew better would know better than to volunteer.
<escowles>i've got to jump on another call, dropping off11:59
* ksclarke_ leaves12:00
<bseeger>ajs6f: speaking of contextualizing, you made me think of this: http://xkcd.com/1640/12:01
<whikloj>awoods: So going forward if you share the google document where people with the link can only "comment", they still have the ability to use suggestion mode for edits.
<ajs6f>bseeger: "When we try to pick out anything by itself, we find it hitched to everything else in the Universe." —John Muir12:02
<awoods>whikloj: that is great12:03
<bseeger>ajs6f: John Muir was a great photographer. ;)
<ajs6f>bseeger: Quite photogenic, anyway: http://digitalcollections.pacific.edu/cdm/search/collection/muirphotos12:05
<bseeger>ajs6f: quite the beard. Never really seen a photo _of_ him before. Nice digital collection as well.12:07
<ajs6f>bseeger: I am always stunned when I find someone doing anything interesting with digital repository technology. I normally assume that it's not possible to do that.12:08
<bseeger>ajs6f: why's that?12:09
<ajs6f>bseeger: Because I have lots of experience with digital repository technology.
<barmintor>because everything is horrible
<ajs6f>barmintor: Such a Negative Nelly. Honestly.12:10
<barmintor>most things are horrible
<barmintor>related: in retrospect, Nelly was horrible12:12
also sunshine++
<ajs6f>barmintor: Were you formerly confused about that?12:15
<barmintor>I try to be flexible
<ajs6f>speaking of which, I'm off to the gym. bbl
* ajs6f leaves12:16
* yinlin leaves13:04
* bseeger leaves13:09
* bseeger joins13:13
* bseeger leaves13:15
* bseeger joins13:24
<jrgriffiniii>Does anyone happen to know of the next Asynchronous Storage Meeting? It appears that they last met on 11/11/15 (https://wiki.duraspace.org/display/FF/2015-11-11+-+Asynchronous+Storage+Meeting), and I can find nothing within Confluence detailing any meetings held between 12/01/15 - 02/01/16.13:31
It also appears that they may have integrated with API-X (?)13:32
* anarchivist leaves13:35
* anarchivist joins13:36
* yinlin joins
* anarchivist leaves13:40
* anarchivist joins13:42
* diegopino joins13:50
<awoods>jrgriffiniii: re: async-storage meeting... would you mind sending an email to Randall Floyd with those questions (and cc fedora-tech). I believe he is working on the next meeting soon.13:51
<jrgriffiniii>awoods: Certainly, I shall address this immediately
<awoods>jrgriffiniii: at your leisure ;)13:52
* dwilcox leaves14:19
* ajs6f joins15:09
barmintor:tjohnson: I think this "is the repository entirely connected via LDP containment" question is actually pretty important. It's going to come up again and again.15:11
<ajs6f>tjohnson: Right now my kneejerk reaction is that I want the repo to be connected. But when I think about it, I want that to happen so that people can walk through it RESTfully. Is that actually how things work? Most people have indexes. But then, there is Google.15:12
<tjohnson>i'm inclined to defer to others on it. I admit to not knowing how it might interact with other object repository expectations15:13
ajs6f: yeah, discovery through interconnectedness seems like a weak reason to restrict it15:14
a good use case, but not a terribly good motivation for telling implementers what they can't do :)15:15
<ajs6f>tjohnson: Yeah. And goodness knows I'm not especially averse to telling implementors what they can't do.15:16
awoods: Have you made a page for next week's call?15:18
The more I think about it, the more I think multi-rooted repositories are just multi-tenancy, which is an affordance for which we have been repeatedly asked.15:23
<tjohnson>ajs6f: not *just* multi-tenancy, though. With arbitrary PUT contain, you could have a free-floating LDP-RS or LDP-NR15:24
<ajs6f>tjohnson: Isn't that just a very small tenant?15:25
<tjohnson>i guess you could look at it that way :)
<ajs6f>tjohnson: I'm not trying to force the idea into that box. I was really just noting that my use case for connectedness was small and weird, but the use case for unconnectedness is large and popular.15:26
Well, _a_ use case for unconectedness.
<ajs6f>tjohnson: But is that enough say that we want to REQUIRE PUT…?15:27
<tjohnson>probably not
<ajs6f>tjohnson: No, I agree. Need more use cases… feed the beast.
<tjohnson>q: does Fedora advertise where its base container lives?
are clients expected to already know it's at `/rest`?15:28
<ajs6f>tjohnson: No, I don't think we advertise in any way except by recursing on has-parent.
Which isn't really advertising at all, I suppose.15:30
<tjohnson>okay. Derby's base is currently just `/`; I think it's reasonable to expect the client to know where the endpoint is. For the multi-tenancy use case, you'd presumably benefit from a way to find the various available endpoints on a server
<ajs6f>tjohnson: Yes, good point. A table of contents. No, more than that, because you can't walk around to all of the partitions in the repo.15:31
tjohnson: Or, not? Maybe for the multitenancy case, knowing where all the roots are is an admin function you don't normally want available?15:32
<tjohnson>ajs6f: 👍15:33
seems like the best idea is to punt the whole thing; per barmintor's original suggestion15:34
<awoods>ajs6f: https://wiki.duraspace.org/display/FF/2016-02-18+-+Fedora+Tech+Meeting
<ajs6f>tjohnson: It may be, but what I worry about is different repos doing PUT, and doing it differently (w/ containment w/o containment). That could get really confusing.15:35
<tjohnson>though i'm in favor of "if resource creation with PUT is supported, repositories SHOULD ______"
<ajs6f>awoods: thanks
the big downside of the no-containment approach is that you need to know a resource doesn't already exist to create it.15:36
<ajs6f>tjohnson: Yes, that may be all we can do, and that would assuage my fears. Not my fears about global warming or how I am going to pay for my children's education, but smaller, more managebale fears.
<tjohnson>if all you know is the URI and the desired state, making PUT act just like POST will save you a HEAD request
<ajs6f>tjohnson: Like a force-overwrite?15:37
<tjohnson>right. as a client, i have a local representation of resource X, and I want to sync that to the server15:38
* dwilcox joins15:39
<ajs6f>tjohnson: RIght, I can see how that promotes efficiency.
<tjohnson>might prefer to do that with PUT for two reasons: because i don't track `exists?`; or because i don't want to implement special logic for create
<ajs6f>tjohnson: This is all true for NRs, too, I suppose. That worries me a bit because I think about weird persistence where even HEAD could be expensive.15:40
* bseeger leaves15:41
<tjohnson>tradeoffs: they are everywhere15:42
* bseeger joins15:45
<ajs6f>tjohnson: True dat. As much as possible I want to find the side of the trade from which Fedora practitioners will benefit more. Of course, part of this APi specing exercise is exactly about finding the limits of our certainty about what will actually beneift Fedora-ifers.15:47
* diegopino leaves15:49
* yinlin leaves16:00
<tjohnson>ajs6f: of course, all of the above depends on the server not requiring ETags. I'm not sure whether Fedora currently rejects PUT updates with no ETag... ?
<ajs6f>tjohnson: I do not know. I _think_ it does not… awoods?16:01
<awoods>tjohnson: no, F4 does not fail if PUTs do not include etags.16:02
<ajs6f>tjohnson: and what the ref impl does not is far from the end of the story, as you know. The whole question of ETag is something I think we could take another look at in the spec context.
<tjohnson>+1. I think i would propose simply following: https://www.w3.org/TR/ldp/#h-ldpr-put-precond
but there may be good reasons to want more clearly specified behavior16:03
awoods: +1. RDF::LDP/Derby are in line with that, then
<ajs6f>tjohnson: I _think_ that's pretty much where we are now with the ref impl. If you use 'em , they have to make sense (match). awoods has referred to this as a kind of optimistic locking.
* dhlamb leaves16:04
<tjohnson>i think it's a good approach, since it allows lightweight clients that don't care about locking
<ajs6f>tjohnson: yes, yes. And locking is… well, acoburn is positively allergic to it at this point, because of his concerns around distributed impls.16:06
<tjohnson>e.g. client == tjohnson + raptor + curl
<ajs6f>tjohnson: Well, client could be some JS app in someone's browser. I would think of that as pretty lightweight too.
<tjohnson>(tjohnson doesn't have extra space in-memory for etags, or exists? flags)16:07
yeah. there are lots of reasons you might not be worried about update races, etc...
<ajs6f>tjohnson: The more I hear from middleware groups like Islandora and hydra, the more I think the solid 80% of use doesn't see much concurrent work on shared resources. There _are_ exceptions (there's an islandora use case where several people are editing the same TEI document) but by and large...16:09
<tjohnson>ajs6f: re: acoburn's concerns: I'm still not convinced they can't be abstracted away at a lower layer than LDP. He's going to need to support ETags one way or another to conform to that spec, too.
<ajs6f>tjohnson: Yeah, but weak ETags can be a lot easier, but you can't necessarily use them in awoods' pattern of "optimistic locking".16:11
tjohnson: I haven't thought about the layering in a distributed impl, but I think you are assuming that there is a layer beneath LDP that would understand this kind of concern. I wouldn't be surprised if there is in some Hadoop impl, but it is an assumptnio.16:12
tjohnson: But I just haven't thought very much about it, because I am not nearly as interested in distribution as acoburn.
<tjohnson>ajs6f: yeah. Getting true atomic updates (e.g. in the face of a database crash) seems out of reach; but getting serializable requests seems possible16:13
*maybe* more expensive than it's worth, but possible
<ajs6f>tjohnson: Indeed. I think here though you have to know a lot about the impl. And we don't, cause it don't exist yet! :)16:14
tjohnson: I mean, what's expensive? It depends. It could even depend on network topology. What's in your wiring closet?
<tjohnson>yes. it will be good to see one in progress to nail these issues down a bit
<ajs6f>tjohnson: Yes. I think acoburn is most interested in that, but I'm sure there are other folks.16:15
awoods: Are the SCAPE people no longer interested in a distributed Fedora?
<tjohnson>ajs6f: I think worst case is LDP-layer locking on edits--i.e. a POST to a container locks the created resource and the container; a PUT to a resource locks the resource; for the duration of the request16:16
<ajs6f>tjohnson: Sure, but there's still a lot inside that. Distributed locks? What when you start locking from the West Coast and me from the East Coast, you know the sort of thing I'm talking about. It's not trivial. Banks and stock markets hire really smart people to work on this stuff.16:17
<tjohnson>my point is mainly that it's not distributed locks at the database level, but distributed locks at the webserver level16:18
which is much easier
and particularly: a problem you already have16:19
<ajs6f>tjohnson: Mn. I'm not sure I agree that it's much easier. Even if we say that it is, that doesn't mean it is easy. :) In any event, I think we can write the specs to avoid too many assumptions about implicit atomicity. If people want that kind of semantics, we are going to offer it to them in a very explicit way.
<tjohnson>👍 to that, in general16:20
<ajs6f>You can't be sincere without being explicit.
And Fedora is all about sincerity.
<tjohnson>and, of course, _transaction_ atomicity is a different beast. pessimistic locking at the webserver layer is clearly not an acceptable solution there16:22
ajs6f: is your feeling that transactions in general should be optional?16:23
<ajs6f>tjohnson: There are going to be some interesting solutions to that in the distributed cases.
tjohnson: If you mean full aCID, I think we established that we don't ven know what that really means for Fedora. I do think we can and must require some kind of atomic durable batch operation.16:24
* dwilcox leaves
<ajs6f>tjohnson: I haven't yet heard anyone offer a use case for isolation… UNSW seems to possibly have one, but they have been ahrd to reach about this.16:25
<tjohnson>ajs6f: i just meant the transaction endpoint, and its services. I think there's something to be said for requiring it and punting ACID semantics to the impl16:26
<ajs6f>tjohnson: But lots of people came up with the need for atomic batch operations. Which kind of sucks, because that is very nonREST kind of gesture.
re: isolation: Read Uncommitted is a bad place to live
<ajs6f>tjohnson: I don't think the endpoint is gong anywhere. I think the semantics from the spec will be atomicity and durability. If an impl wants to do more (the ref impl now does isolation) great.
<tjohnson>i assume implementations will want to do isolation at at least the Read Committed level, yes?16:27
<ajs6f>tjohnson: Again, there are question about how it all works smoothly in distribution. Not saying it can't be done, I (and acoburn) are saying that we don't want impling a distributed Fedora to require someone to write a PhD diss.
<tjohnson>agreed there16:28
I'm increasingly convinced that beyond durability and corruption-level consistency, there's not much the spec should require16:29
<ajs6f>tjohnson: yeah, I think for sanity's sake impls will do read commited isolation. But you know I can imagine "embedded" patterns of usage that _might_ not even require that. If there's a single app/client, and it is managing concurrency in some other way...
* bseeger leaves
<tjohnson>actually, I'm not committed to durability (see: Derby) :)
<ajs6f>tjohnson: Are you excluding atomicity?
<tjohnson>for transactions, I think so16:30
<ajs6f>tjohnson: Well, durability is realtive. Derby show durability relative to a read ten minutes later, just not ten days later. :)
<tjohnson>right. Durability until restart
<ajs6f>tjohnson: Hm. I want to cut off as much as possible, but I think that (losing atomicity) would cause an uproar.
tjohnson: Really. Not a single use case we got for transactions didn't assume atomicity.16:31
<tjohnson>i'm not closed to it; I'm just not assuming it as minimal16:32
i mean, I was willing to make the whole transaction endpoint optional :)
<ajs6f>tjohnson: Well, the spec process (from a cerrtain POV) is about what is minimal.
<awoods>ajs6f: SCAPE seems to be taking a backseat... using their own impl until something proves better.
<ajs6f>tjohnson: You and me both. But if you go back and look at the original documents around Fedora 4 (then called Fedora Futures) the concern about transactions was there from the beginning.16:33
<tjohnson>and I think acoburn's concern is really about atomicity. i.e. that updating a contained resource & the container would need to be a two step process, happening on different hardware
<ajs6f>tjohnson: Well, each of those pieces (updating contained and container) could be on N pieces of hardware. Some of which might be at great latency. I think of use cases from APTrust or DPN.16:34
tjohnson: Lots of room for the unexpected...16:35
<tjohnson>right. that's even worse (re: atomicity) than the impl acoburn was spitballing in San Diego last week. :)
<ajs6f>tjohnson: Was he showing something over HBase? That's the last thing I heard him mention.
* dwilcox joins16:38
* dwilcox leaves16:40
<tjohnson>ajs6f: just talking about it over drinks, but yeah HBase16:45
<ajs6f>tjohnson: Could be cool. We looked at Hadoop early in the Fedora 4 effort. We didn't go there because of the horrible effort it takes to manage it. Even just to get a test running against it was painful. Maybe it's better now.16:46
* bseeger joins
* dwilcox joins16:51
<ajs6f>later all16:52
* ajs6f leaves
* bseeger leaves16:54
* kat3_drx leaves16:58
* kat3_drx joins
* kat3_drx leaves
* dwilcox leaves16:59
* mikeAtUVa leaves
* jrgriffiniii leaves17:17
* mohamedar leaves17:18
* jcoyne leaves17:22
* jgpawletko leaves17:30
* github-ff joins17:37
[fcrepo4-vagrant] awoods created fcrepo-1876.2 (+1 new commit): https://git.io/vgiAP
fcrepo4-vagrant/fcrepo-1876.2 f2ba7af Andrew Woods: Update README.md
* github-ff leaves
* github-ff joins
[fcrepo4-vagrant] awoods opened pull request #34: Update README.md (master...fcrepo-1876.2) https://git.io/vgiAD
* github-ff leaves
<ruebot>awoods: https://github.com/fcrepo4-exts/fcrepo4-vagrant/pull/34 -- you good with that? i'm ready to merge if so.17:38
<awoods>ruebot: yes... just some minor readme updates17:39
<ruebot>awoods: cool. good stuff :-)17:40
* github-ff joins
[fcrepo4-vagrant] ruebot pushed 1 new commit to master: https://git.io/vgixR
fcrepo4-vagrant/master 4ca2583 Nick Ruest: Merge pull request #34 from fcrepo4-exts/fcrepo-1876.2...
* github-ff leaves
* peichman leaves17:49
* github-ff joins17:50
[fcrepo4-vagrant] awoods deleted fcrepo-1876.2 at f2ba7af: https://git.io/vgihI
* github-ff leaves
* whikloj leaves18:01
* jgpawletko joins18:13
* jgpawletko leaves18:20
* mcritchlow leaves19:12
* jcoyne joins19:32
* jcoyne leaves20:00
* jcoyne joins20:05
* jcoyne leaves20:24
* dhlamb joins21:18
* mcritchlow joins22:18
* jcoyne joins22:37
* dhlamb leaves22:45
* mohamedar joins23:12
* mohamedar leaves23:24
* mcritchlow leaves00:11
* jcoyne leaves00:27