Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* apb18 leaves02:26
* thomz joins02:48
* dchandekstark joins05:08
* dchandekstark leaves05:13
* ajwagner leaves06:21
* ajwagner joins06:23
* dwilcox joins06:54
* dchandekstark joins07:10
* dchandekstark leaves07:14
* dwilcox leaves07:18
* dwilcox joins07:53
* manez joins08:00
* manez leaves08:04
* manez joins08:12
* coblej joins08:36
* dchandekstark joins08:51
* ajs6f joins09:00
* dhlamb joins09:02
* mikeAtUVa joins09:08
* dchandekstark leaves09:14
* jjtuttle leaves09:15
* dchandekstark joins09:16
* acoburn joins09:17
* jjtuttle joins09:19
<ajs6f>ruebot: on your im/ex list, "Support transacting in RDF" doesn't specify packaging or how to deal with bitstreams, but BagIt does. Maybe BagIt comes first?09:20
* ruebot looks09:31
<escowles>ajs6f: ruebot: fwiw, the camel-based rdf serializer does specify those things — not that we'd use that tooling here, but it would be nice if we could follow its conventions and be compatible with it09:32
<ajs6f>escowles: I didn't know that. In that case, we might as well just reuse that immediately to keep momentum.
<ruebot>ajs6f: oh, "Support transacting in RDF" - I interpret that as just dumping RDF to the filesystem, for example ttl.
<ajs6f>ruebot: Yeah, but ^^^ we have to have binaries too, right. But it doesn't matter. See what escowles says.09:33
<acoburn>ruebot: ajs6f: that's what the camel machinery does: rdf and bitstreams to the filesystem
it doesn't handle versioning, though
<ruebot> escowles: yeah. awoods and i talked about that. that whatever tool we create would have the same functionality. we didn't want to require camel.09:34
* dchandekstark leaves
<escowles>ruebot: yep, i had the same discussion with awoods last week — i agree that requiring camel for import/export would be a frustratingly high bar
<ajs6f>I would think we want to somehow add versioning to what is already there.
<ruebot>ajs6f: yep. should we re-word "Support allowing the option to include Binaries"?09:35
<ajs6f>And the important work is not impl, but design. Camel / no Camel is not as important as having a really solid understanding of how you serialize this stuff.
ruebot: What do you mean? Are you thinking of not supporting that?
<ruebot>ajs6f: oh wait, i think i see what you mean; exporting the RDF and associated binaries at the same time, how to do we package that?09:36
<escowles>ajs6f++ completely agree — i hope we wind up with an interactive/non-camel tool, and also update the camel tool to support the same features (maybe even sharing the same core code)
<ajs6f>ruebot: Right.
escowles: Yep, same picture drawn in different media.09:37
Like my son likes to draw using ketchup and mustard, but my daughter prefers blackberries. But they are both drawing the same amorphous blob.
escowles: Sharing code could be tricky. That would be great, if it could happen, but it would require an expensive level of coordination.09:38
<ruebot>ajs6f: ah, good. on the same page. The first step in my mind -- phase one -- would be just dumping it all to the filesystem. Phase two would package it in a BagIt bag.
<ajs6f>escowles:acoburn: Although, doesn't Camel allow to use beans written in scripting languages?
<acoburn>ajs6f: yes09:39
<ajs6f>ruebot: That's fine. I didn't understand until escowles set me right that we already have a form for the work.
<ajs6f>escowles:acoburn: maybe there can be shared code....
<escowles>ajs6f: yes, i don't think sharing code is a short-term goal — but if the import/export tool winds up with usable code, the camel rdf serializer might be updated to use it
<ajs6f>ruebot++ # nice work in that list of priorities!09:40
escowles: +1
<ruebot>ajs6f: thanks!
<ajs6f>ruebot: I feel fully prioritized!
It's like stepping out of a fresh shower into a brisk wind.
* ruebot snorts
<mikeAtUVa>acoburn: regarding the camel serialization, what happens if your updates happen faster than the serialization routine processes the messages?09:41
<acoburn>mikeAtUVa: you should be using a queue, not a topic
<ajs6f>mikeAtUVa: Wait a bit. Wait a bit. Wait a bit. Queueing.
mikeAtUva: See what I did there?
<mikeAtUVa>acoburn: does a queue gurantee synchronous processing?09:42
* apb18 joins
<ajs6f>mikeAtUVa: Specifically not.
<acoburn>mikeAtUVa: in a messaging context, nothing is synchronous
<ajs6f>acoburn: Hold on. You can certainly use synch workflows in a messaging context. They are the limiting cases of asynch workflows.
<ruebot>escowles, barmintor: have either of you given thought as to what the tool would be written in? curious if we'll have a jar, or expand on josh's python work.09:43
<acoburn>ajs6f: ok, sure, but in the context of Fedora's messaging, no
<mikeAtUVa>acoburn: ajs6f: forgive my ignorance... but imagine you change dc:title 5 times a second and your serialization only processes 4 times a second, when it serializes, wouldn't it lose versions (ie, make serialization through camel impossible to include history)
<acoburn>mikeAtUVa: this depends on the broker configuration
mikeAtUVa: the default configuration is for Fedora to send messages to a topic, and yes, messages can get dropped09:44
mikeAtUVa: in any *real* production scenario, you should be using a queue
<escowles>ruebot: i've been assuming java was the most likely (given the rest of the fcrepo ecosystem), but i'm willing to consider other things depending on who's doing the work
<mikeAtUVa>acoburn: I'm not talking about dropped messages, just messages that are processed after the thing they relate to (an old dc:topic) is no longer present in the repository.
<escowles>i'd be much more useful in ruby or java, but would be fine with python if that makes sense09:45
<acoburn>mikeAtUVa: it is entirely possible, especially if you have a lot of updates followed by a DELETE
<ruebot>escowles: i'm assuming java too; that'll limit me in a development role. but, as always i'll help out where i can, or worst case, learn something :-)
<escowles>ruebot++ # yeah, i'm happy to be a tester and learn some python if that's the way we go09:46
<mikeAtUVa>acoburn: that what I thought, and it (in my mind) severely limits the utility of those types of integrations for critical applications (ie, journaling and so forth)09:47
acoburn: it worked for fedora 3 because the processor could always request what a resource looked like at the point the message was emitted. We could probably build a similar version-aware serialization that listened to version creation events and then requested that version for serialization.09:48
<ajs6f>mikeAtUVa: I don't see the problem. What you see at the APi is the "sum" of all the events emitted. Why would that be a problme?09:49
<acoburn>mikeAtUVa: as long as the downstream application operations are idempotent, there is rarely a problem here09:50
<ajs6f>Sorry, "product" is the term I should have used
mikeAtUVa: Are you worried about getting all of the versioins serialized?
<mikeAtUVa>ajs6f: here's a simple example that would be impossible because of the issue I'm disscussing... say you wanted to listen to camel events and build a mirror repository that has the same history of updates (albiet with slightly different timestamps).09:51
<ajs6f>mikeATUva: And?09:53
<acoburn>mikeAtUVa: that would be a difficult thing to build with the current (and planned future) Fedora messaging interface
mikeAtUVa: i.e. assuming you wanted incremental sparql-update PATCH operations on the mirror
mikeAtUVa: and not PUT operations
<ajs6f>mikeAtUVa:acoburn: I don't see why… you get a message, you go and retrieve the versioing that messga creating, and you a diff to create the patch. Not triial, but perfectly doable.
You're going to get minimal updates instead of exactly the same updates, but is that bad?09:54
<mikeAtUVa>acoburn: ajs6f: these aren't revelations, I know, but when the camel infrastructure is touted as a possible solution for export/restore, I feel like I'm missing something.
<acoburn>ajs6f: yes, that's doable and definitely not trivial
<ajs6f>mikeAtUVa: It does nothing for restore. It's all about export. Think of it as fine-grained export, fine-grained in time, not resource. Every time you make a change, it records it _just_ the same way you would export it, so that when it comes time to do your big export, you don't have to. It's just the sum of all the little exports that Camel has been doing the whole time.09:55
<acoburn>ajs6f: in my experience, it's just a lot easier to use idempotent operations b/c order doesn't matter as much, which would exclude PATCH
<ajs6f>acoburn: I agree entirely, but I was trying to speak to his example.
<mikeAtUVa>ajs6f: except it doesn't... not if you process the messages more slowely than the updates.09:56
<ajs6f>mikeAtUVa: Yes, it does, if you take that fact into account when you do your export operation.
mikeAtUVa: It's not just a simple GET. There's some logic to it.09:57
You have to GET (and serialize) a version.
got to run to a meeting
<mikeAtUVa>acoburn, ajs6f: thanks for trying to help educate me... it's helpful, but I have to run to a meeting now.
* dwilcox leaves
<mikeAtUVa>^^^ same meeting...
<ajs6f>It's a sign that we need to document this design idea better.09:58
* ajs6f leaves
* peichman joins
* github-ff joins10:08
[fcrepo-webapp-plus] awoods opened pull request #48: Enable inheritance of logging levels (master...fcrepo-2109) https://git.io/v6lLn
* github-ff leaves
* dwilcox joins10:13
* escowles leaves10:54
* esm_ joins
* escowles joins
* thomz leaves
* dchandekstark joins11:01
* dchandekstark leaves11:05
* bseeger joins11:06
* manez leaves11:12
* manez joins11:16
* dhlamb leaves11:20
* ajs6f joins11:22
* dhlamb joins11:32
* coblej leaves11:56
* github-ff joins12:03
[fcrepo4-vagrant] ruebot pushed 1 new commit to 4.6.0-RC: https://git.io/v6luN
fcrepo4-vagrant/4.6.0-RC f641730 Andrew Woods: Add configuration support for AuthZ and no-AuthZ (#52)...
* github-ff leaves
* ajs6f leaves
* acoburn leaves12:10
* dchandekstark joins12:20
* dchandek_ joins12:22
* dchandekstark leaves12:24
* dchandek_ leaves12:26
* github-ff joins12:28
[fcrepo4-vagrant] awoods opened pull request #53: Add configuration support for AuthZ and no-AuthZ (master...fcrepo-2110-master) https://git.io/v6lV7
* github-ff leaves
* github-ff joins12:30
[fcrepo4] awoods opened pull request #1090: Add repository configuration for AuthZ (master...fcrepo-2110) https://git.io/v6lwm
* github-ff leaves
* dchandekstark joins12:38
* acoburn joins13:00
* esm_ leaves13:09
* esm_ joins13:11
* esm_ leaves13:23
* esm_ joins13:28
* escowles_ joins13:55
* escowles leaves13:59
* esm_ leaves14:02
* dchandekstark leaves
* dchandekstark joins14:03
* apb18 leaves14:33
* dchandekstark leaves15:01
* dchandekstark joins15:07
* esm_ joins15:14
<f4jenkins>Yippee, build fixed!15:25
Project fcrepo4 build #3366: FIXED in 10 min: http://jenkins.fcrepo.org/job/fcrepo4/3366/
* dchandekstark leaves15:50
* esm_ leaves16:00
* dchandekstark joins16:05
* dchandekstark leaves16:06
* dchandekstark joins
* dhlamb leaves16:11
* manez leaves16:16
* ajs6f joins16:24
* dwilcox leaves16:25
* acoburn leaves16:33
* ajs6f leaves16:39
* dwilcox joins16:49
* mikeAtUVa leaves16:53
* peichman leaves17:11
* bseeger leaves17:22
* adur leaves17:31
* github-ff joins17:35
[fcrepo4] ruebot pushed 1 new commit to master: https://git.io/v68an
fcrepo4/master ffff6a4 Andrew Woods: Add repository configuration for AuthZ (#1090)...
* github-ff leaves
* travis-ci joins17:57
fcrepo4/fcrepo4#4661 (master - ffff6a4 : Andrew Woods): The build passed.
Change view : https://github.com/fcrepo4/fcrepo4/compare/1833da04b382...ffff6a4f1ab0
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/151347353
* travis-ci leaves
* dchandekstark leaves17:59
* dwilcox leaves18:31
* dchandekstark joins18:55
* dchandekstark leaves19:00
* dwilcox joins19:28
* dwilcox leaves19:32
* dchandekstark joins20:50
* github-ff joins21:38
[fcrepo4] acoburn closed pull request #1087: Enable inheritance of logging levels (master...fcrepo-2109) https://git.io/v6Z1x
* github-ff leaves
* github-ff joins21:39
[fcrepo-webapp-plus] acoburn closed pull request #48: Enable inheritance of logging levels (master...fcrepo-2109) https://git.io/v6lLn
* github-ff leaves
* travis-ci joins21:46
fcrepo4-exts/fcrepo-webapp-plus#155 (master - 26b2e16 : Andrew Woods): The build passed.
Change view : https://github.com/fcrepo4-exts/fcrepo-webapp-plus/compare/ed728f342018...26b2e16cf362
Build details : https://travis-ci.org/fcrepo4-exts/fcrepo-webapp-plus/builds/151391744
* travis-ci leaves
* travis-ci joins21:55
fcrepo4/fcrepo4#4662 (master - b0538ba : Andrew Woods): The build passed.
Change view : https://github.com/fcrepo4/fcrepo4/compare/ffff6a4f1ab0...b0538bafae39
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/151391531
* travis-ci leaves
* dchandekstark leaves22:13

Generated by Sualtam