Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* rlefaive leaves05:31
* rlefaive joins06:40
* awead joins07:27
* dwilcox joins07:39
* dwilcox leaves08:09
* dwilcox joins08:10
* ajs6f joins08:18
* dhlamb joins08:22
* rlefaive leaves08:53
* rlefaive joins08:57
* rlefaive leaves08:58
* rlefaive joins09:02
* rlefaive leaves09:08
* umgrosscol joins09:09
* jgpawletko joins09:11
* rlefaive joins09:12
* acoburn joins09:14
* acoburn1 joins09:21
* acoburn leaves09:24
* acoburn joins09:28
* acoburn1 leaves09:30
* yinlin joins09:34
* yinlin leaves09:35
* whikloj joins09:38
* github-ff joins10:37
[fcrepo-camel-toolbox] acoburn opened pull request #59: decouple build version (master...fcrepo-1728b) http://git.io/vc2jW
* github-ff leaves
<ajs6f>awoods: This RDA thing seems like a distraction Fedora can ill afford. The chief attraction of each spec and standard with which we are now engaged (LDP, WebAC, perhaps Memento, etc.) is that none is a parochial construction of the academic environment.10:39
<awoods>ajs6f: That is a point well-taken. I think it will be interesting, nonetheless, to engage in the initial RDA conversation to see what they have in mind... maybe, just maybe, they have an interest in the same services/standards that we care about.10:43
<ajs6f>awoods: That's a fair point, too. If the participants willing to countenance the simple fact that institutions of memory are not special unique snowflakes, they'll have a chance to do something useful. Are you trying to put together some kind of "official" representation from Fedora?10:47
<awoods>ajs6f: Actually, dwilcox, would you mind giving some more context?10:48
<ajs6f>dwilcox: Would you please contextualize the living snot of it?10:49
<dwilcox>awoods ajs6f : On a call but I will respond shortly10:51
<ajs6f>At least awoods issn't on a call, thank goodness.
<awoods>ajs6f: are you on a call?10:52
<whikloj>ajs6f: Is your concern that we already have a baseline of services (ie. LDP) and this will not add anything useful to that set?
<ajs6f>awoods: I am always on the call in my mind.
<whikloj>sorry, are you on a call?
<awoods>whikloj: in his mind10:53
<f4jenkins>Project fcrepo4-T2 build #375: UNSTABLE in 5 min 16 sec: http://jenkins.fcrepo.org/job/fcrepo4-T2/375/
<whikloj>awoods/ajs6f: it's where I have all my best conversations
<ajs6f>whikloj: No, my concern is that what these people will get about will be the creation of repository-software-specific APIs that do the same things as APIs like LDP or WebAC, except poorly and with the support of much smaller communities of use.
<whikloj>ajs6f: So a mish-mash of subsets of existing APIs.10:54
<ajs6f>The conference call in my mind is always running, and yet somehow everyone is always on hold waiting to join, and the hold music is better than the best music anyone has ever heard in reality.10:55
whikloj: No, exactly the opposite: freshly minted APIs that run over the same ground that existing APIs already cover perfectly well.
whikloj: _Libraries do this all the freakin' time._10:56
<whikloj>ajs6f: agreed
<ajs6f>whikloj: awoods makes the point that such need not be the outcome, which is true, but history affords us no sanguinity in this matter.10:58
<whikloj>ajs6f: maybe European universities will be different and freely accept the existence of prior APIs
* github-ff joins
[fcrepo4] awoods pushed 1 new commit to master: http://git.io/vcaLq
fcrepo4/master a532284 Yinlin Chen: Add link to registry on splash-page...
* github-ff leaves
<ajs6f>whikloj: European institutions, in my small experience, by and large, _do_ tend to be a little better about this.
whikloj:awoods: BUT… https://rd-alliance.org/please-share-september-16-webinar-highlighting-rda-outputs-and-endorsers.html10:59
* github-ff joins
[fcrepo4] awoods closed pull request #915: Splash-page link to registry (master...FCREPO-1593) http://git.io/vcuBY
* github-ff leaves
<ajs6f>Most of those documents offer _nothing_ new.
* awoods waiting on context from dwilcox... it is going to be awesome!11:00
* github-ff joins11:02
[fcrepo-camel-toolbox] awoods closed pull request #59: decouple build version (master...fcrepo-1728b) http://git.io/vc2jW
* github-ff leaves
<ajs6f>When dwilcox contextualizes, it's like listening to Richard Feynman lecture on physics or Kool Herc spin discs. You suddenly realize what greatness is.
<whikloj>perhaps instead of a call with the RDA we should organize a spoken-word night with dwilcox?11:03
<dwilcox>awoods / ajs6f : RDA is structured around interest and working groups, and the primary participants are not librarians but people who manage research data or develop/maintain research data repositories, many of which are domain-specific with regard to metadata standards, data models, and file formats. Recognizing that we'll never get everyone in the world to adopt a common data model and/or repository platform, this group is interested in11:08
figuring out how to achieve some level of interoperability between existing repositories (particularly in the context of research data). The initial thought was that a generic API could be the way to approach this, but as Andrew and I have discussed standardizing on import/export formats might be a better approach.
In any case, this is an initial call to discuss the effort in more detail before actually writing a case statement and forming the group
It is not platform specific, so the desire is not to have Fedora represented specifically11:09
Rather, members of this community have clearly thought about repository interoperability for some time, so I thought it would be useful to have their input. Lessons learned, ideas about how best to proceed, etc.
<ajs6f>dwilcox: I'm familiar enough with RDA to have noticed that their output suffers from pretty much the same problems as library-based groups. I agree that im/ex makes much more sense, but that is clearly not what the group's explicit purpose is. Is it your contention that that purpose might be changed?11:10
<awoods>ajs6f: can you expand "im/ex"?11:11
<ajs6f>awoods: Sorry, "import/export formats", as mentioned by dwilcox.
got it
<dwilcox>ajs6f: Yes - this group does not yet exist. 6 or 7 of us got together in Paris at Wolfram's request to discuss this as a formative idea. A generic API is just the current idea on how to approach this - the problem we're trying to solve is interoperability between disparate repositories. An API is one approach, but it may not be the best one
<ajs6f>awoods: NOT http://www.exim.gov/authority-has-lapsed/11:12
<dwilcox>I think the group is amenable to other ideas at this point
But that is just my impression
<ajs6f>dwilcox: Okay, that makes it marginally more palatable to me. I would certainly be willing to show up and make the best argument I could to discard nebulous dreams of API internationalism in favor of practical rules for an extensible serialization scheme.11:13
HTTP is the API. What's actually needed is a set of representation forms.11:14
<dwilcox>ajs6f: I think that would be valuable input.
I am eager to ensure this group embarks on the right quest
* travis-ci joins11:15
fcrepo4-exts/fcrepo-camel-toolbox#162 (master - 2013b95 : Andrew Woods): The build passed.
Change view : https://github.com/fcrepo4-exts/fcrepo-camel-toolbox/compare/5544b9bac73a...2013b95e002e
Build details : https://travis-ci.org/fcrepo4-exts/fcrepo-camel-toolbox/builds/83320173
* travis-ci leaves
<awoods>(right quest)++
<ajs6f>whikloj: Was that not some seriously powerful context? Does that not leave you feeling a little uplifted, a little bigger than you were?
<dwilcox>ajs6f: I am flattered to be compared to a luminary like Kool Herc11:16
<ajs6f>dwilcox: In the right light, you even look like him a bit. I mean, less black and less Jamaican, but other than that.11:17
* travis-ci joins
<dwilcox>ajs6f: Minor differences.
<travis-ci>fcrepo4/fcrepo4#4074 (master - a532284 : Yinlin Chen): The build passed.
Change view : https://github.com/fcrepo4/fcrepo4/compare/45ae114d27b4...a532284d45e5
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/83319385
* travis-ci leaves
* rlefaive leaves11:25
* rlefaive joins11:29
* dwilcox leaves11:30
* rlefaive leaves11:33
<whikloj>ajs6f: that was some awesome context.11:34
ajs6f: the man is a legend. http://goo.gl/2REYJk
* rlefaive joins11:38
<ajs6f>whikloj: I'm really inspired to put some kind of dwilcox/KH easter egg into the code.11:39
* github-ff joins11:42
[fcrepo-camel-toolbox] acoburn opened pull request #60: added URL encoding support, related to upstream fcrepo-camel changes (master...fcrepo-1752) http://git.io/vca4n
* github-ff leaves
* dwilcox joins11:43
<whikloj>ajs6f: http://niobe.lib.umanitoba.ca/islandora/object/uofm%3A1246970
<ajs6f>whikloj: Beautiful. Let's use that as the basis of a test object for F4 ITs.
* barmintor joins11:49
<dwilcox>whikloj: The resemblance is closer than I would have thought11:53
<ajs6f>dwilcox: You can really see the similarity in your beats and breaks.11:54
<whikloj>dwilcox: It is uncanny and until I hear a set I believe this project manager gig might be a cover.
* dhlamb_ joins
<barmintor>ajs6f: relevant to list pain of the week-11:55
<dwilcox>whikloj: I don't believe we've ever been seen together in the same place...
<barmintor>ajs6f: documenting the ways that our instituional understanding of APT as preservation storage has obscured the amount of work required of us to deposit11:56
<whikloj>dwilcox: exactly
<ajs6f>barmintor: eh? You mean that thinking of APT as simple persistence hid the fact that it isn't?
<barmintor>ajs6f: yes
<ajs6f>barmintor: Are you coming down to DC this week/
<barmintor>ajs6f: "They use BagIt! We use BagIt! It'll be fine."11:57
* dhlamb leaves
<barmintor>ajs6f: no, but I'm writing a brief dossier up for my boss.
<ajs6f>barmintor: This is what I want dwilcox to say on the committers' list thread— that the RDA group could be interested in ideas of serialization instead of vague API handwaving.11:58
<barmintor>ajs6f: not only is APTrust trying to be a repository, but the material requirements of ingest/deposit matter (and are not "API" per se)11:59
* github-ff joins
[fcrepo4] awoods pushed 1 new commit to master: http://git.io/vcazk
fcrepo4/master 6ea28b2 Aaron Coburn: Replace regex with jena request parser...
* github-ff leaves
<ajs6f>barmintor: Also, it's always a pleasure to hear dwilcox rock the floor.
* github-ff joins
[fcrepo4] awoods closed pull request #913: replace regex with jena request parser (master...fcrepo-1706) http://git.io/vnxF9
* github-ff leaves
<ajs6f>barmintor: Are you talking about specific bitstream formats (like image formats)?
<dwilcox>barmintor: We had that discussion here earlier today - I am on board with suggesting a different approach to generic APIs, like import/export formats or whatever makes the most sense12:00
<ajs6f>barmintor: make no mistake, APT _is_ a repository. If you don't like that, you'd better start talking loudly and lengthily to a lot of people.
<barmintor>ajs6f: No, like "because deposit is an actual thing and not an abstraction, your deposits need to be smaller than X, and TAR files, and transmitted to Y"
ajs6f: it's less a matter of not liking it, and more a matter of conditioning expectations12:01
<ajs6f>barmintor: Oh, are you saying that APT isn't doing a good job of explaining how ingest and integration is going to / does work?
<barmintor>ajs6f: tho I also would contend that it's only sort of a repository, b/c stating that a Bag is a digital object doesn't get you very far12:02
ajs6f: No, no- docs are improving. Just that the accident of deposit process has implications.
<ajs6f>barmintor: Agreed, but that's not where it's supposed to end. Years ago Greg Jansen and I agreed that the only way APT becomes interesting is if you integrate the semantics over the contents. Otherwise it's just a buying club for cloud backup storage.12:03
barmintor: You're seeing pressure from APT workflow backing up into your systems? Having effects on your processes?
<barmintor>ajs6f: but the horse gets out of the barn, you know? If I upload 5T of content I'm not really inclined to say "delete all that, I'm reuploading with better semantics"12:04
ajs6f: yeah- once we depart from "this apt-object is a whole collection of columbia-objects", you have to do some work.12:05
<ajs6f>barmintor: Sure, sure. We had hoped that APT would become the place to _create_ those better semantics, using the power of the aggregation. I'm not saying that it's a sure bet, or that every single person involved with APT sees it this way.
<barmintor>ajs6f: hence my quest to stream virtual TAR files to S3 this week12:06
<ajs6f>barmintor: Hm. I think I see your point. Do you have specific suggestions about how to relax/alter constraints on APT workflow to avoid those kinds of pressures? Some of those constraints may come from APT impl choices, and could be altered...
<barmintor>ajs6f: I don't know. I think some of them are inevitable.12:07
* dhlamb joins12:08
<barmintor>ajs6f: we have work upcoming to change from our workflow from "upload content on preso storage" to "export from our repo system to APT"
<ajs6f>barmintor: I am going to Washington Monday, and I would be happy to buttonhole others to talk about this, if I was confident that I could offer concrete suggestions. Of course, I can always go down the hall and talk to Chip or Andrew Diamond, but a partner meeting is a chance for other institutions to say, "Yeah, we have that problem too."
barmintor: So do we. I don't think that's inherently problematic, as long as we all get recompensed for the work in the form of better guarantees, costs, and affordances.
barmintor: Is Rob coming?12:09
<barmintor>ajs6f: some of it, too, is that until I have had the time to hash out these basics, I can't see the changes we need to make
ajs6f: IDK Stephen is.
<ajs6f>barmintor: Well, okay— early days. I'm sure mdurbin here feels the same way.
barmintor: I'm hopeful, so my reaction is to try to figure out what to do to make sure we're getting value for that bother you describe. What can we get out of the aggregation that we just couldn't get out of our own stuff?12:10
* dhlamb_ leaves
<barmintor>ajs6f: I mean, one of my early complaints was "we don't have enough spare storage to stage the bags", but I have hacked my way around that now
<ajs6f>barmintor: Yeah, honestly, that doesn't seem like a deal-breaker...12:11
<barmintor>ajs6f: this is the assessment of someone who has enough storage to stage the bags
<ajs6f>I don't have enough bookshelving in my house, but that doesn't stop me from buying books.
* barmintor laughs
<ajs6f>barmintor: Batching?
<barmintor>ajs6f: initially I had thought to upload our extant Bag, but they're too big. Now I have client libraries identify an object and stream a more limited bag to S312:12
<ajs6f>barmintor: Why are your Bags so fat? (Take that however you like.)12:13
* osmandin joins
<barmintor>ajs6f: we store deposits to preservation as Bags. They're big, because they're "Preservation Images for Exhibition X"12:14
<f4jenkins>Yippee, build fixed!
Project fcrepo4-release-tests build #360: FIXED in 1 min 28 sec: http://jenkins.fcrepo.org/job/fcrepo4-release-tests/360/
<barmintor>ajs6f: for this week, we shifted from "upload all the newspaper archive" to "upload each issue in the newspaper archive"12:15
<ajs6f>barmintor: Ah, I see. Well, this is going to be completely unhelpful, but I think you're doing it wrong. You really are just using APT as really costly cloud backup.
<barmintor>ajs6f: yes, this is the aforementioned opinion12:16
<ajs6f>barmintor: Right, issues not archives. Why not just go with the granularity you are using for your own purposes?
barmintor: How does getting more fine-grained make you require _more_ tmp?
barmintor: Shouldn't it be the opposite?12:17
<barmintor>ajs6f: in the repository, both the preservation deposit and the issues are represented
ajs6f: but again- these aren't being exported from the repository system yet
<ajs6f>barmintor: But you are duplicating the _bits_?
<barmintor>ajs6f: we were, in the TAR file12:18
<ajs6f>barmintor: What affordance to yourselves is that supporting?12:19
barmintor: Disaster recovery from your own repo?
<barmintor>ajs6f: I don't understand your question
ajs6f: the bags-on-disk?
* travis-ci joins
<ajs6f>barmintor: Why would you do that? Why duplicate the bits, then send them to APT?
<travis-ci>fcrepo4/fcrepo4#4075 (master - 6ea28b2 : Aaron Coburn): The build passed.
Change view : https://github.com/fcrepo4/fcrepo4/compare/a532284d45e5...6ea28b2d707d
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/83331931
* travis-ci leaves
* osmandin leaves
<ajs6f>barmintor: Actually, why duplicate the bits inside your own repo at all?
<barmintor>ajs6f: I don't duplicate the bits in my system. I duplicate the bits in the TAR file I am required to upload12:20
<ajs6f>barmintor: Oh, okay. And then you upload it, and then you delete the TARfile, right?
<barmintor>ajs6f: yes
<f4jenkins>Project fcrepo-camel-toolbox build #226: UNSTABLE in 7 min 34 sec: http://jenkins.fcrepo.org/job/fcrepo-camel-toolbox/226/
<barmintor>ajs6f: the TAR file is temporary
ajs6f: but also unpredictably big12:21
<ajs6f>barmintor: And if you are getting finer-grained, then that cycle gets shorter and tmp storage can be smaller, right?
<barmintor>ajs6f: yes, but even better is just not store the TAR file at all
<ajs6f>barmintor: Sure, I get that. Some kind of dynamic flow. Are you saying that APT isn't letting that happen because of their ingest workflow?12:22
<barmintor>ajs6f: I'm saying that an ingest requiring a TAR file of <= 250G means you have to do work to conform; more work that "rsync a directory" for example12:23
<ajs6f>barmintor: Yes, what I'm trying to get at is _why_ you have to do the work: I'm not that familiar with APT's reqs. Do you have to send TAR files?12:24
<barmintor>ajs6f: and also that this kind of work is inevitable, and not a modeling concern but an accidental concern of the very act of depositing
ajs6f: yes
ajs6f: you have to transmit a BagIt archive, that unpacks to an appropriately named directory, as a TAR file, to S312:25
ajs6f: and the Bag has to have an md5 payload digest, and it has to have APT tags in the bag-info
<ajs6f>barmintor: Okay, now we're getting somewhere that I can actually take to DC. So you would want, ideally, APT to rectify their workflow to avoid quantizing transmissions/ingestions? You'd like to see, say, rsync or GridFTP be a usable tool?
<barmintor>ajs6f: well, last week maybe12:26
ajs6f: it's all water under the bridge now
ajs6f: but when Stephen describes what I have done, feel free to suggest these things
ajs6f: but this gets back to the email!12:27
ajs6f: repository notions have consequences.
<ajs6f>barmintor: Maybe for you, but it will come up again and it's probably coming up for other people. That's why I'm belaboring this ahead of Monday. I'd like partner institutions to have a chance to hear this and say "Gosh, so _that's_ what my devs were complaining about."
<barmintor>ajs6f: and some of those consequences are not in *the model* but in the friction around *actual process*12:28
ajs6f: and (of course) there will always be an actual process.
I hope12:29
<ajs6f>barmintor: Sure, they do. No argument here. I do want dwilcox to contextualize that RDA invitation, because from what he said earlier today in this channel, the inviters are not as committed to League of Nationism as that invite would make it sound.
I could hug you, Adam
<ajs6f>barmintor: Don't, you'll get the cilantro mayonnaise I'm having with this flounder all down my shirt.12:30
<dwilcox>ajs6f barmintor : I replied to the email thread with some context - let me know if more would be helpful
Hmm, actually it looks like I don't have permission to post to the committers list12:31
I should probably get awoods to add me to the list
<ajs6f>barmintor: I'm looking at dwilcox's words: "The initial thought was that a generic API could be the way to approach this, but as we have discussed standardizing on import/export formats might be a better approach."12:32
barmintor: I think talking about serializations has _some_ value.
<barmintor>ajs6f: this I agree with
<awoods>dwilcox: what list?12:33
<dwilcox>awoods: fcrepo-committers
<awoods>dwilcox: why is that?12:34
<dwilcox>awoods: I tried to reply to the thread and got a message saying I don't have permission to post to the list
<ajs6f>barmintor: You— you agree with me?! {Sniff} I— I never thought it could be like this...12:35
<barmintor>ajs6f: you had me at "cilantro mayonnaise"12:36
<awoods>dwilcox: try posting now...
<ajs6f>barmintor: Now raising a glass of homemade cider to all the poor bastards down in the trenches shoveling bits...12:37
<dwilcox>awoods: That appears to have worked
<awoods>dwilcox: now the anyone can post to the list... but the message is moderated.12:38
<dwilcox>awoods: Great
<ajs6f>awoods: You mean that if post another raging screed, it will actually be rewritten into a toned-down and objective version.12:42
<awoods>ajs6f: give it a shot...12:43
<ajs6f>awoods: Wow, you're right. What I sent featured hellacious cursing and a incredibly inappropriate picture of two donkeys doing unmentionable things with a DSpace repo. But it all came out quite reasonable sounding.12:47
<barmintor>ajs6f: I wish I had a time lapse video of me responding to one of your emails I could share with you.12:48
<awoods>ajs6f ;)
<ajs6f>barmintor: I guess it starts out like this:12:49
* github-ff joins12:53
[fcrepo-karaf] acoburn opened pull request #13: add maintainer section (master...add_maintainers) http://git.io/vcaQS
* github-ff leaves
* jgpawletko leaves13:11
* github-ff joins14:20
[fcrepo-camel] awoods pushed 1 new commit to master: http://git.io/vcVCd
fcrepo-camel/master 9fde12a Aaron Coburn: URL-Encode output...
* github-ff leaves
* github-ff joins
[fcrepo-camel] awoods closed pull request #95: URL-Encode output (master...fcrepo-1752) http://git.io/vct94
* github-ff leaves
* github-ff joins
[fcrepo-camel-toolbox] awoods closed pull request #60: added URL encoding support, related to upstream fcrepo-camel changes (master...fcrepo-1752) http://git.io/vca4n
* github-ff leaves
* travis-ci joins14:23
fcrepo4-exts/fcrepo-camel#230 (master - 9fde12a : Aaron Coburn): The build passed.
Change view : https://github.com/fcrepo4-exts/fcrepo-camel/compare/6b161c28eb6f...9fde12a6d90f
Build details : https://travis-ci.org/fcrepo4-exts/fcrepo-camel/builds/83354882
* travis-ci leaves
* rlefaive leaves14:29
* travis-ci joins14:32
fcrepo4-exts/fcrepo-camel-toolbox#164 (master - 2053e07 : Andrew Woods): The build passed.
Change view : https://github.com/fcrepo4-exts/fcrepo-camel-toolbox/compare/2013b95e002e...2053e073b444
Build details : https://travis-ci.org/fcrepo4-exts/fcrepo-camel-toolbox/builds/83355015
* travis-ci leaves
<f4jenkins>Yippee, build fixed!14:34
Project fcrepo-camel-toolbox build #228: FIXED in 6 min 37 sec: http://jenkins.fcrepo.org/job/fcrepo-camel-toolbox/228/
* dwilcox leaves
* rlefaive joins14:36
* github-ff joins15:14
[fcrepo4] yinlinchen opened pull request #916: Version creation should return 201 (master...FCREPO-1755) http://git.io/vcVPp
* github-ff leaves
* jrgriffiniii joins15:51
* github-ff joins15:53
[fcrepo4] awoods closed pull request #914: Do not expose UUID of versions - HTML UI (master...FCREPO-1715) http://git.io/vc0yT
* github-ff leaves
* dhlamb leaves16:11
* ksclarke joins16:22
* dwilcox joins16:32
* dhlamb joins16:43
* dhlamb leaves16:48
* dwilcox leaves16:56
* acoburn leaves17:29
* ksclarke leaves17:31
* jrgriffiniii leaves17:55
* whikloj leaves17:57
* awead_away leaves17:59
* ajs6f leaves18:16
* dhlamb joins18:45
* dhlamb leaves18:49
* rlefaive leaves18:58
* the_mgt_ joins19:49
* the_mgt leaves19:53
* umgrosscol leaves21:15
* dhlamb joins21:20
* ksclarke joins21:44
* ksclarke leaves21:48
* ksclarke joins21:49
* ksclarke leaves21:53
* ksclarke joins22:12
* ksclarke leaves22:16
* ksclarke joins22:26
* ksclarke leaves22:35
* ksclarke joins
* ksclarke leaves22:36
* ksclarke joins22:37
* ksclarke leaves22:41
* ksclarke joins22:42
* dhlamb leaves23:00
* f4jenkins joins23:10