Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* github-ff joins03:57
[fcrepo4] barmintor created agg-fixity (+1 new commit): http://git.io/CSwbtg
fcrepo4/agg-fixity e9d6ec0 Benjamin Armintor: aggregate fixity
* github-ff leaves
* github-ff joins03:58
[fcrepo4] barmintor opened pull request #36: aggregate fixity (master...agg-fixity) http://git.io/VBGQ-A
* github-ff leaves
* barmintor leaves03:59
* nbanks leaves06:05
* kaarefc joins06:33
* nbanks joins06:34
* nbanks leaves06:44
* nbanks joins06:50
* eddies leaves06:55
* eddies joins07:02
* eddies leaves
* eddies joins
<ff-pivotal>Edwin Shin edited "Configure Infinispan to store multiple copies of the data in different directories" https://www.pivotaltracker.com/story/show/4569056907:15
Edwin Shin edited "Extend "RepositoryService" to handle performing fixity checks and return a status for each cache store" https://www.pivotaltracker.com/story/show/4600325707:16
Edwin Shin added comment: "This won't work w/ Jetty." https://www.pivotaltracker.com/story/show/4595002707:17
Edwin Shin edited "Investigate possibilty of deploying kitchen-sink as an EAR" https://www.pivotaltracker.com/story/show/45950027
Edwin Shin edited "Deploy fcrepo-fixity-webapp to fcrepo4.fcrepo.org" https://www.pivotaltracker.com/story/show/45952901
Edwin Shin edited "Deploy fcrepo-fixity-webapp to fcrepo4.fcrepo.org" https://www.pivotaltracker.com/story/show/4595290107:18
Edwin Shin added "Fedora 3 to 4 Migration Utility" https://www.pivotaltracker.com/story/show/4601810707:20
* eddies leaves08:18
* eddies joins08:35
* eddies leaves
* eddies joins
<ff-pivotal>A. "Schwarmerei" Soroka edited "Convert fcrepo-generator-rdf to use any23" https://www.pivotaltracker.com/story/show/45864223
* VincentNG joins08:44
<ff-pivotal>Vincent Nguyen finished "Repository size should not go negative" https://www.pivotaltracker.com/story/show/4496264308:59
* ajs6f joins09:11
cbeer— have you seen this:09:19
https://docs.jboss.org/author/display/ISPN/Infinispan+Distributed+Execution+Framework
?
We could use this for async fixity...
Or maybe even better:09:21
https://docs.jboss.org/author/display/ISPN/Map+Reduce+framework
* ajs6f leaves
<ff-pivotal>Vincent Nguyen delivered "Repository size should not go negative" https://www.pivotaltracker.com/story/show/44962643
* ajs6f joins09:26
<cbeer>ajs6f: i knew it existed, but haven't read the docs09:41
<ajs6f>I'm catching up and just reading your new fixity code.
<cbeer>hm, ok, i think i can see how to use the execution framework here09:47
<ajs6f>Yeah, that ones a little more straightforward.
It doesn't require framing into the MapReduce abstractions, just Runnables.
<cbeer>at least, it answers my next question, of how to check each member of the cluster
<ajs6f>Yeah, and parallelization is a good thing.
cbeer— I'm still trying to understand why we absotutely need to go to ISPN. Clearly, using parallelization is good, but are we actually using anything to calculate fixity we can't get from the BinaryValue (within the MODE context)?09:48
<cbeer>ajs6f: we want to get the fixity of a datablob in EVERY cachestore; I believe the BinaryValue will just give us the fixity in ANY cache store.09:50
this might only be relevant in a ChainedCacheStore scenario, if the DEF stuff plays out for clustering though
<ajs6f>So it's about replication, which is not visible from MODE?
<cbeer>yes. and, to some extent, I guess, asynchronous storage.09:51
<ajs6f>k, got it.
<cbeer>(although I haven't looked at async storage yet)
<ajs6f>Welll, replication alone means we have to go to IPSN.
<cbeer>but you want to be able to ask the question "hey, has this blob arrived at AWS yet" or whatever
<ajs6f>Quick style question, how come LowLevelStorageService @Injects a Repository and casts it to a JcrRepository in a static method? Why not just @Inject the JcrRepository?09:55
<cbeer>probably because I didn't know we could do that?
<ff-pivotal>Chris Beer edited "Fixity checks against every instance of a data blob in a distributed-cache mode" https://www.pivotaltracker.com/story/show/4562802509:57
Chris Beer edited "Extend the RepositoryService content retrieval to support clustering." https://www.pivotaltracker.com/story/show/4599635709:58
<ajs6f>Oh, actually, you had to do that, because you're using an injected resource from static methods. Oh, well.10:04
<cbeer>(i'm not sure why they are static, though. the other *Service were, so I made this one static too)10:06
<ajs6f>The way I've been playing it in Services, methods are static. They're basically helper classes with their only real state being their read-only Session.10:07
This makes them easy to use by clients. But if we need more state, we should think about them becoming injectable beans.10:08
VincentNG, you did the new views stuff, right? +1. Do you know if Velocity can return InputStreams instead of Strings?10:09
* kaarefc leaves10:11
<ajs6f>cbeer: am I right in thinking that LowLevelStorageService concerns itself only with the MODE Binary store?10:15
<cbeer>it's a little trickier than that, but, yes, that's all I set out to tackle.10:16
<ajs6f>Make sense. We might want to change some method names just to make that clear.
<cbeer>fine with me. i /think/ the binary store is the only kind of store we care about exactly where the data is coming from10:17
<ajs6f>Well, my concern is that people might want to work fixity on metadata, which might be in non-Binary properties (like Strings or relationship properties).10:19
<cbeer>hm
yeah, i don't want to think about supporting that yet.10:20
* github-ff joins10:22
[fcrepo4] ajs6f pushed 1 new commit to master: http://git.io/l81W_A
fcrepo4/master 0baf2dd ajs6f: Cleaned up unused imports
* github-ff leaves
<ajs6f>No, me neither. Just wanted to set a bit in the code to remind us that the current service ignores that stuff.10:23
cbeer— is StoreChunkInputStream basically buffering a chunked return from a Cache?10:24
<cbeer>yes. stolen almost wholesale from modeshape, tweaked to work against a single cache store.10:25
<ajs6f>k got it.
<cbeer>(because modeshape chunks its binaries)
i think there's a lot of undesirable assumptions present in that code, btw. probably worth having a discussion upstream at some point10:26
(especially the, "hey, i didn't find a chunk. guess that means i'm at the end" assumption )
<ajs6f>Right, right. Yeah, I could see using something like a Reader wrapper for the CacheEntry-s and then wrapping a bunch of them with something like com.google.common.io.MultiReader.10:29
<cbeer>makes sense.10:31
* kaarefc joins10:32
* barmintor joins10:34
<ajs6f>cbeer— Is the ISPN-aware LowLevelStorageService being exposed in fcrepo-http-api yet?10:35
<cbeer>ajs6f: i think barmintor's pull request does that
<ajs6f>Superb...
<cbeer>https://github.com/futures/fcrepo4/pull/3610:36
<barmintor>this pattern of using a singleton bean to do static class initialization, and then using static imports from the class to approximate mixins- is that a thing? It's clever, but it kind of terrifies me (spent some time hunting down a NPE last night)10:37
I also can't offer a better way to do it :)
<ajs6f>That is the pattern I was going for (see comment above from about ten minutes ago).
I agree, it's ugly.
Scala does it much better. {grin}10:38
<cbeer> Oh, actually, you had to do that, because you're using an injected resource from static methods. Oh, well.
cbeer (i'm not sure why they are static, though. the other *Service were, so I made this one static too)
ajs6f The way I've been playing it in Services, methods are static. They're basically helper classes with their only real state being their read-only Session.
<barmintor>it's like an anti-pattern voltron
<cbeer>This makes them easy to use by clients. But if we need more state, we should think about them becoming injectable beans.
<ajs6f>It's weird, yes. I'm not proud of it, but it's quick and flexible if the service backplane continues to comprise a small number of simple, orthogonal services.10:39
<barmintor>cbeer: that reminds me, one of the service mixins has a instance-member session
which I think is an error?
* barmintor looks
<cbeer>DatastreamService
<ajs6f>barmintor— was that what produced the NPE?
<cbeer>clearly it's not using it.10:40
(barmitor: the NPE was not having the bean declared in spring?)
<barmintor>ajs6f: the service bean not being loaded was what was producing it- it's just not what you expect from static imports :)
<cbeer>(or, please tell me i'm not the only one who spent an hour trying to track that down)
<barmintor>cbeer^^10:41
<ajs6f>Right, right. That _is_ maybe the ugliest part.
Again, I'm not proud of it. I'm happy to go to something better.
<barmintor>I'm just glad I have someone to blame ;)
<cbeer>hello ff-pivotal10:42
hello
<ff-pivotal>Hello, cbeer
<ajs6f>The reason that the state is there at all is so that people can offer *Service an identifier(s) and get a live resource from the repo. That requires a Session, and I chucked that into the *Service-s to avoid creating and destroying them over and over and over all over the place.
Patches welcome...10:43
:)
<barmintor>ajs6f: the result of it is really pretty cool. It just sets off my spidey-sense.
(and also NPE)
<ajs6f>You're right. It's funky. Sneaky state hiding behind static methods.
<barmintor>Another hack would be to have the repobeanfactory intialize the service base class, and have them al share a read-only session10:44
<ajs6f>Subterranean state sneaking inside a static surface.
barmintor: ++110:45
There is no base class now, but that would be trivial. Want to make me a ticket?
<barmintor>ajs6f: I don't know- it's realy just trading frequency of ugly for increase degree of ugly10:46
<ajs6f>I don't think so. I think it maintains the level of ugly but concentrates in one easier-to-track location.
And it lightens the deadweight of Spring XML.10:47
<barmintor>"one easier-to-track location" <- famous last words
<ajs6f>Well, maybe I just need to do some hard thinking about a really good pattern. A plain singleton might be the right archetype here.
<barmintor>I have to get coffee and get to the interviews. (I'd kind of miss the mixin approximation- it's really nice when it's working)10:48
<ajs6f>I'll leave it be for now and moot any changes before moving on them.
* pivotal-bot joins10:56
* pivotal-bot leaves
<VincentNG>ajs6f: Sorry, i was in a meeting. But regarding your question about velocity, i don't think Velocity can return an inputstream but i'm not positive10:57
<ajs6f>Cool. No prob. I was just curious.
* travis-ci joins11:00
[travis-ci] futures/fcrepo4#209 (master - 0baf2dd : ajs6f): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/a0fe42903155...0baf2dd6021c
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/5442135
* travis-ci leaves
<cbeer>barmintor: can you tell VincentNG the status of https://www.pivotaltracker.com/story/show/45628025?11:03
<ajs6f>cbeer: Trying to get my head around LowLevelStorageService.getBlobs() to respond to your request to tighten LowLevelStorageService. Is the expected result there a Map with an entry for every Cache but nulls as values for those Caches that don't have a copy of the bitstream11:04
?
<cbeer>sounds right, I guess.. TRUE/FALSE/FILE_NOT_FOUND as always.11:05
<ajs6f>:)
<VincentNG>AFK11:09
<cbeer>guava is pretty nice
google++
<ajs6f>Aye. It brings functional-ish niceness. The Function<> and Predicate<> make the lack of closures less agonizing11:10
And it's tight, efficient, and ultra-well-tested.
<cbeer>ajs6f: ok, maybe i'm just being dense but,.. is there a guava way to go from a List to a map, with the key being the value from the list and the value being the result of a Function/EntityTransform?11:12
<ajs6f>Hm. Haven't needed one, so not sure. I _am_ sure that they've taken a lot of care for composability. Give me a minute or three and I'll cook it up for you.11:13
* kaarefc leaves11:14
<cbeer>ok, i've found Multimap.index, which is the other way around
and maybe i'm just supposed to use that
<ajs6f>Hm.
<cbeer>(and, for my purposes, that might be fine..)
(and, if not.. invertFrom seems to be a thing)11:15
<ajs6f>that plus invertFrom(), eh?
Great minds think alike.
Also, if you end up with a MultiMap<K,V>, one way to "flatten" it for which I know some of the methods is by going to Map<K,Collection<V>>.11:19
And then picking or concatenating the values.
<cbeer>hm: https://code.google.com/p/guava-libraries/issues/detail?id=56
<ajs6f>v14 came out in February. We can update if you want that.11:21
<cbeer>nah, i'm willing to stop that line of questioning.11:24
<ajs6f>Probably we ought to update anyway.
I'll try it just for fun.11:25
<cbeer>ok, merging barmintor's pr11:26
* github-ff joins11:27
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/JjQ8tQ
fcrepo4/master fa4c455 Chris Beer: Merge branch 'master' into agg-fixity...
* github-ff leaves
* github-ff joins
[fcrepo4] cbeer deleted agg-fixity at e9d6ec0: http://git.io/R5QHFw
* github-ff leaves
* github-ff joins11:28
[fcrepo4] ajs6f pushed 3 new commits to master: http://git.io/BqVbLw
fcrepo4/master 6e4b198 ajs6f: Small improvement to LowLevelStorageService
fcrepo4/master f325b70 ajs6f: Small improvement to LowLevelStorageService
fcrepo4/master d15d16b ajs6f: Merge branch 'master' of github.com:futures/fcrepo4
* github-ff leaves
<cbeer>ajs6f: ok, i ran into this the other day and ended up cheating.11:41
<ajs6f>What?
<cbeer>so, there's the DigestInputStream that calculates the digest for me
is there something similar for getting the content size?
<ajs6f>Like a ContentSizeInputStream?
<cbeer>yeah11:42
<ajs6f>There is in Commons IO.
Let me find it.
http://commons.apache.org/proper/commons-io/apidocs/org/apache/commons/io/input/CountingInputStream.html
I don't think there is in Guava, but it might be worth looking.
<cbeer>thanks.. i guess i'm creating my own FixityInputStream or something to mash them together nicely11:43
<ajs6f>The usual pattern is to wrap.
<cbeer>yup
<ajs6f>But you could package the wrapping into your own class for convenience.
<cbeer>exactly my plan.
oh, i remember the other context11:44
<ajs6f>No one likes seeing a lot of new X (new Y (new Z)))
even functional goons.
<cbeer>when writing new datastreams, we want to get the size and digest too
and i punted and just let modeshape tell me what happened
<ajs6f>Do we not trust MODE, or do we just want to keep it indpendent, or… I mean, I'm cool with it. Especially for digest.11:45
<cbeer>ajs6f: i was happy to trust MODE then, but it seems strange to track fixity, but trust the first hop into the system implicitly11:46
<ajs6f>True that
We must maintain the chain of evidence, or the crooks will get off in court!11:47
<cbeer>is this on the right track? https://gist.github.com/cbeer/777736b8e8c53faa828712:01
<ajs6f>cbeer— I changed the contract of LowLevelStorageService.applyDigestToBlobs() to return a map of <Cache,InputStream> for the successful validations, which seems lcloser to what barmintor suggested, but probably it needs improvement.12:02
* nbanks leaves
* github-ff joins
[fcrepo4] ajs6f pushed 1 new commit to master: http://git.io/dBjKmw
fcrepo4/master d8739c1 ajs6f: More improvements to the implementation of LowLevelStorageService
* github-ff leaves
<barmintor>ajs6f: I don't follow you
isn't that what getBlobs does?12:03
<cbeer>wait, InputStream?
<ajs6f>No, getBlobs just gets blobs.
This gets you the stores and blobs that _are_ valid.
It does all the fixity checking.
InputStream is the stand in for the blob.
<cbeer>i think you want all the stores, though
<ajs6f>Why?12:04
<barmintor>https://github.com/futures/fcrepo4/blob/master/fcrepo-kernel/src/main/java/org/fcrepo/services/LowLevelStorageService.java#L119
* travis-ci joins
[travis-ci] futures/fcrepo4#211 (master - d15d16b : ajs6f): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/fa4c455b8416...d15d16bd390c
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/5443896
* travis-ci leaves
<barmintor>Oh, are you saying it gives you a filteres list of the valid blobs?
<ajs6f>barmintor— sure. That does no fixity checking.
* travis-ci joins
[travis-ci] futures/fcrepo4#210 (master - fa4c455 : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/0baf2dd6021c...fa4c455b8416
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/5443856
* travis-ci leaves
<ajs6f>Yes, filtered. Filterd by whether it passes fixity check.
<cbeer>eddies wants to be able to fix up the bad ones.
<ajs6f>"bad ones"?
<cbeer>so we need to be able to identify them, no?
<ajs6f>Oh, you want healing.
<cbeer>datastreams where the fixity check failed.
<ajs6f>Like Wolverine.12:05
<barmintor>that seems like not what we want- don't we need to know about failures besides inferring it from hte number of returned inputstreams?
<cbeer>and i think we want to be able to report:
data in stores A, B, C passed
<ajs6f>How about another method that tells you what failed.
<cbeer>data in store D was a total mess.
<ajs6f>?
So what are the contracts? Let's list them...
(not worried about the names right now, just the sigs and semantics)12:06
<eddies>+1 to "and i think we want to be able to report: data in stores A, B, C passed. data in store D was a total mess."
<barmintor>getBlobs(BinaryKey) : returns all the blobs
<ajs6f>Check.
getValidBlobs(BK)?
<barmintor>getFixity(BinaryKey): returns all the fixity statuses12:07
<ajs6f>getInvalidStoresForBlob()?
* github-ff joins
[fcrepo4] cbeer created fixity-blobs (+1 new commit): http://git.io/dNZstA
fcrepo4/fixity-blobs 575c38e Chris Beer: update applyDigestToBlobs to return a FixityResult object with the checksum and size
* github-ff leaves
<ajs6f>FixityReport getFixity(BinaryKey): returns all the fixity statuses
<cbeer>that branch is where i thought we were going. but maybe i'm wrong
i'll let you guys figure it out while i head to the office :)
<ajs6f>I didn't even know that branch existed.
<cbeer>we were working concurrently,12:08
<ajs6f>Ah, concurrency is tricky, unless you are Rich Hickey.
So do we have a FixityResult abstraction?
<ff-pivotal>Edwin Shin added "Demonstrate pointing fcrepo4 on top of an external data source" https://www.pivotaltracker.com/story/show/46039601
<cbeer>that's what I made, yes. it's not a FixityStatus because FixityStatus has some jaxb stuff in it
but they might be unifiable.12:09
<ajs6f>Actually, I would prefer FixityResult<T> with T something like FedoraObject or Datastream
Because we might want different kinds of reports.
<cbeer>do we have a plan to calculate fixity on jcr nodes?
(or, i don't know what fixity of an object looks like.)12:10
<ajs6f>The fixity of the object, for me, could start with the fixity of the JCR-spec'd serialization of it.
Not perfect, but someplace to start.
I'm sitting in a meeting for APTrust at which it has become extremely clear that there is an appetite for object-level fixity.12:11
<barmintor>you mean a list of the fixity checks on all the associated datastreams?12:12
<ajs6f>No, I mean a fixity check on the persistence of the object structure and metadata, but it could include an overall check on the datastreams. There's interest in both.12:13
But the overall check sounds like more of a convenience, since we have individual checking.
<barmintor>:|
<ajs6f>What's yer beef?
<barmintor>it makes total sense in FCR3, where the object has one serialized structure. Though I don't know where you store the fixity info.12:14
it makes less sense in a JCR context.
<ajs6f>I don't see why not… howsoever the repo stores the stuff, we still want to know that it's doing it right. My idea to use the JCR serialization is just a first pass— I'm happy to use something better out of the gate.12:15
<barmintor>but what are you fixity-checking?
<ajs6f>Metadata on the object and relationships to other things (mostly objects).12:16
<barmintor>(I still have the question about where you stick the fixity info for a foxml file)
<ajs6f>Dublin Core, for example.
<barmintor>that's a datastream, right?
<ajs6f>The stuff we really don't want to provide special datastream slots for any more.
No, it's not, according to cbeer.
Nor me.12:17
It _could_ be, if you choose.
<barmintor>you want to store fixity information on the jcr properties?
<ajs6f>good question.
Hm.
<barmintor>this is what I'm saying
<ajs6f>No, I'd rather store it elsewhere in the repo.
Or in the external store that was proposed the other day.12:18
<barmintor>I think al you can really do is have some algorithmic serialization, and stick that checksum somewhere
but it still seems weird to me12:19
<ajs6f>That would be fine with me, for now, and if there are issues with it, we can worry about them then. There already is an algo. serial., carefully specified by JCR and implemented by MODE.
<barmintor>maybe it's not weird. but it seems very fragile.
oh, well if it's a solved problem by JCR than that's that.12:20
<ajs6f>Yeah, it's spec'd. Let me find the API...12:21
<eddies>i can see ajs6f's case coming more into play when we build that external, filesystem preservation service. there you want fixity checks on everything that gets serialized out. maybe you try to use that to go back and validate jcr property values in Mode… (but this last bit sounds weird as i type it out :P)12:22
<ajs6f>Yeah, that makes sense. Listen, I'm not going to freak out about it now, any more than I'm already kind of a freak. I just wanted to get the thinking going a little.12:23
Just for completeness:12:24
http://www.day.com/specs/jcr/2.0/7_Export.html
<barmintor>that might not allow for fixity12:26
<ajs6f>You mean putting it in there? No, I doubt it would. I meant, that's the serialization that you can use to _create_ fixity.
<barmintor>unless it is specified that the export serialization is identical every time it's run
<ajs6f>Mm, I thought it did, but I'll check that. Depending on what the exact contract is, we could use XML Signature or something like that. Or maybe that just doesn't work and I would need to think about how to obtain the repo-internal footprint of a given FedoraObject in a sensible way that could be used to fixity.12:28
<barmintor>in some ways, it's analogous to the way you verify checksums: the actual checksum is an array of bytes, and you just have to know how that array was represented as a string that you're going to compare it against12:29
<ajs6f>I'm not going to worry a huge amount about it now. Like I said, this is more about the use case that I just heard. But APTrust won't be moving onto fcrepo4 anytime soon.
<cbeer>while we're discussing that, is it safe to restore the non-filtering, fixity-calculating applyDigestToBlobs from my branch?
* barmintor is worrying about it! a huge amount!
<ajs6f>cbeer— please. I clearly misunderstood badly what the needs were. But you might take a look at what I did if you want to see how to use Guava for that kind of thing, which I remember being your original question.12:30
barmintor— really?12:31
I'm starting to think that you are sometimes being facetious.
<barmintor>ajs6f: !!!
Actually, I worry about it a little
* nbanks joins
<ajs6f>Well, have a glass of something delicious.
<barmintor>b/c APT is a source of much use case
<ajs6f>Yeah, but we've got a man on the scene, to keep track of things.12:32
<barmintor>but he does it in scala. SCALA!
* kaarefc joins12:50
<eddies>ajs6f: is there a software output for APTrust that is public? e.g. a github repo?12:51
i'm sort of blissfully unaware of aptrust's activities except from tidbits i hear from you =)12:52
* travis-ci joins
[travis-ci] futures/fcrepo4#212 (master - d8739c1 : ajs6f): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/d15d16bd390c...d8739c1cef6e
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/5444973
* travis-ci leaves
<eddies>screenshot of pretty fixity reporting courtesy of frank, btw: https://docs.google.com/a/yourmediashelf.com/file/d/0B5nd_qlYdcqyTHdXVHpyNWdRZTg/edit12:54
<ajs6f>eddies— just talking about that kind of thing. I'll bring it up with Scott Turnbull (tech lead).12:56
Also, SCALA!
<eddies>BASIC!
<ajs6f>APL!
No, wait. No APL.12:57
<ff-pivotal>Edwin Shin accepted "Build a service to reach behind Modeshape and pull out an InputStream to the raw, low-level storage of a binary." https://www.pivotaltracker.com/story/show/4599633313:01
* github-ff joins13:02
[fcrepo4] cbeer pushed 2 new commits to master: http://git.io/GZSOuA
fcrepo4/master b5102cc Chris Beer: fix up stuff
fcrepo4/master 7951f90 Chris Beer: take another stab at .getFixity; let the consumer decide what to do with the fixity information.
* github-ff leaves
<ff-pivotal>Edwin Shin added comment: "What commit was this fixed in? (i.e. can someone link to it here?)" https://www.pivotaltracker.com/story/show/44962643
<cbeer>ok, maybe that's a better idea of where i thought we were going.
s/maybe/hopefully13:03
* ajs6f leaves13:04
<ff-pivotal>Vincent Nguyen added comment: "https://github.com/futures/fcrepo4/commit/18014998a5c780b721b29ac005f27332c60dfaf0
sorry about the white..." https://www.pivotaltracker.com/story/show/44962643
<cbeer>probably help if i fixed up downstream consumers too13:07
cbeer--
* ajs6f joins13:08
<ff-pivotal>Benjamin Armintor added comment: "https://github.com/futures/fcrepo4/commit/e9d6ec09a947dc7ce2ca9f3837153e33eb5ee7cb" https://www.pivotaltracker.com/story/show/45628025
Benjamin Armintor edited "Fixity checks against every instance of a data blob in a distributed-cache mode" https://www.pivotaltracker.com/story/show/45628025
* github-ff joins13:10
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/xSQWrw
fcrepo4/master 9e7976c Chris Beer: fix up FedoraDatastreams to use new getFixity method
* github-ff leaves
* nbanks leaves13:11
<ajs6f>eddies, Scott told me to look at https://github.com/APTrust for some code.13:13
eddies: also, he and I thought it might be a nice idea for you and he to have a 'phone call and talk through some of the use cases he thinks APTrust will be bringing forth in the near term. He's very excited to make APT a platform for testing/proving out fcrepo4's scalability.13:16
<eddies>ajs6f: i'd be happy to do that
<ajs6f>Groovy. (Not the language, but we should _totally_ rewrite the whole kernel in Groovy. Just to make it hurt.) I'll send y'all an email to put you in touch.13:17
<eddies>ajs6f++13:19
<ajs6f>++ to the suggestion about Groovy?! :)13:20
<barmintor>you know what I love about JCR? No type 'X'13:21
<cbeer>or 'E' or 'R'.13:23
<ajs6f>We have mooted the possibility of MODE's federation supporting the functionality of 'E".13:24
<cbeer>in a hand-wavy sort of way.
PATCHES WELCOME
<ajs6f>I can patch the issue tracker, as long as you patch the code. :)13:25
<eddies>I did just add a story for MODE federation =)13:26
i actually think it sounds like a pretty awesome feature. where "sounds like" is the key assumption13:27
<barmintor>where would I have recently stubbed out the minimal FOXML for a new object in a fcrepo project?13:28
<ajs6f>Darn. I was going to add a story for you to add that story. Seriously, they're pretty proud of it, because it lets you overlay much of the JCR semantics. Their Git example is pretty awesome. If it really works well, which I don't know.
barmintor: I usually end up picking a repo, POSTing to objects/new, then exporting it and cutting it down.13:29
<eddies>ajs6f: https://www.pivotaltracker.com/story/show/4496264313:30
* kaarefc leaves
<ajs6f>eddies: eh?
<barmintor>ajs6f: Hmm. How does that rest call map to the Management module...
<ajs6f>barmintor: objects/new? I think it must go to ingest(), no?13:31
<barmintor>gotta give ingest a foxml inputstream, right?
<ajs6f>I don't know, but I believe you.13:32
<barmintor>https://github.com/fcrepo/fcrepo/blob/master/fcrepo-server/src/main/java/org/fcrepo/server/rest/FedoraObjectsResource.java#L56113:33
<ajs6f>Wow. That's a lot of String literals.
<barmintor>yeah.
<ajs6f>Well, "@author cuong.tran@yourmediashelf.com" is the only author listed, so let's blame eddies.13:34
<barmintor>fortunately we made that method private, so that it can just get copied into other classes that need to do that.
:{
I probably wrote that damned method13:35
<ajs6f>If the FOXML XSD was awesome, you could just have oXygen or another tool build you a minimal valid example.13:36
I don't think that it is. :(
<barmintor>I think that method actually has an encoding bug in it. oh well.13:37
<ajs6f>I have a use case for FOXML in Kilngon.
Klingon.
* barmintor would not be shocked to get an angry email about it13:38
<eddies>yeah, that's some damn ugly code. technically, it pre-dates my formal involvement at mediashelf ;-)
* pivotal-bot joins
<ajs6f>From the Federation Diplomatic Service, working under an EU contract.
<cbeer>hey pivotal-bot: https://www.pivotaltracker.com/story/show/44962643
* nbanks joins
<ajs6f>eddies: I always thought you were one of the founders of MS...13:39
* pivotal-bot leaves
* pivotal-bot joins13:40
<cbeer>pivotal-bot: https://www.pivotaltracker.com/story/show/44962643
<eddies>i think we advertise that i was a founder, but technically, it existed before me
maybe you want to say:
ff-pivotal: https://www.pivotaltracker.com/story/show/44962643
* pivotal-bot leaves
<eddies>oh. there really was a pivotal-bot in the room13:41
i didn't see that
<ajs6f>What's with pivotal-bot? Where did she come from?
* ajs6f leaves
<barmintor>this is awesome: http://muffin.doit.org/docs/api/gnu.regexp.REFilterInputStream.html13:42
<eddies>ajs6f: i assume cbeer is tweaking https://github.com/futures/pivotal-irc (or some fork/branch/etc of it)
barmintor: sarcasm?13:43
* pivotal-bot joins
<cbeer>pivotal-bot: https://www.pivotaltracker.com/story/show/44962643
<pivotal-bot>Repository size should not go negative (owner: Vincent Nguyen)
<barmintor>eddies: No, I think it's great. I don't want to write a filterinputstream
* ajs6f joins
<barmintor>it's like cheap templating13:44
<ajs6f>Actually looks kind of handy.
* pivotal-bot leaves
<eddies>barmintor: i'm so judge-a-book-by-its-cover and couldn't get past the snazzy graphics in the javadocs ;-)
<ajs6f>Except they use some GNU gnu.regexp thing, instead of usual Java Pattern stuff.13:45
* pivotal-bot joins
<cbeer>pivotal-bot: https://www.pivotaltracker.com/story/show/44962643
<pivotal-bot>bug: Repository size should not go negative (delivered) / owner: Vincent Nguyen
<barmintor>OH, :D that's Gnu!
* pivotal-bot leaves
<ajs6f>Have we tested fcrepo4 on Hurd?
<barmintor>they should really have some dancing xmas trees in there
<eddies>i see cbeer is trying to bring up skynet via our irc bot13:46
<ajs6f>eddies: It's the ultimate form of clustering.
* ff-pivotal leaves
* pivotal-bot joins13:49
<cbeer>https://www.pivotaltracker.com/story/show/44962643
<pivotal-bot>bug: Repository size should not go negative (delivered) / owner: Vincent Nguyen
<cbeer>my work here is done.13:50
<ajs6f>Question for all: where did we leave the serialization service questions? IOW, are there tickets for that?
<eddies>i think there are some tickets around serialization in the icebox, but nothing very spec'ed out13:52
<ajs6f>Okay. I'll poke around.
<eddies>https://www.pivotaltracker.com/story/show/44962643asdfasdf13:53
<pivotal-bot>bug: Repository size should not go negative (delivered) / owner: Vincent Nguyen
<cbeer>it's a dumb regex.
<eddies>https://www.pivotaltracker.com/story/show/44962643asdfasdfxxx
<pivotal-bot>bug: Repository size should not go negative (delivered) / owner: Vincent Nguyen
<eddies>=)
<cbeer>PATCHES WELCOME!
<eddies>muhahah
<cbeer>(i just didn't want to get in the business of trying to properly parse a url)13:54
* nbanks leaves
<cbeer>https://www.pivotaltracker.com/story/show/45751509
<pivotal-bot>chore: Add ops to eddies on #duraspace-ff (accepted) / owner: Chris Beer
<eddies>#45751509
<cbeer>nope. thought about it though.
<ajs6f>Nice try, eddies.
<cbeer>anyway, really heading into work now.13:55
* ajs6f leaves13:58
* ajs6f joins14:01
<cbeer>pivotal-bot: assign me a ticket!14:22
* nbanks joins
* github-ff joins14:37
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/maVpsQ
fcrepo4/master 173bd68 Chris Beer: reverse the order of the feed, so newest entries are on top.
* github-ff leaves
* ajs6f leaves14:57
* travis-ci joins15:00
[travis-ci] futures/fcrepo4#214 (master - 7951f90 : Chris Beer): The build has errored.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/d8739c1cef6e...7951f9019856
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/5446589
* travis-ci leaves
* VincentNG leaves15:01
* ajs6f joins
* VincentNG joins
<ajs6f>This looks kind of cool:15:23
https://code.google.com/p/jcrom
I was playing with a strawman for serialization, and I found something annoying. JCR has no way to talk about a Node outside of the context of its repo. For example, I can't find a way to add a Node object to a repo. This could make serialization a lot uglier.15:24
I think I'll have to write something that walks the repo graph and "penetrates" the JCR abstractions to the simpler types that they enclose.15:25
Gross.
It looks like JCROM could avoid all that, but at the cost of reworking a lot of our code (and introducing a new, totally unknown dependency).15:29
* github-ff joins15:30
[fcrepo4] cbeer created webhooks (+1 new commit): http://git.io/c5Rx7A
fcrepo4/webhooks c23bdaa Chris Beer: first stab at repository-wide webhooks
* github-ff leaves
<cbeer>i really don't want to convert that eventbus Event to something pleasant to work with.15:33
maybe i should just listen to the JMS instead.
<ajs6f>If our internal Events aren't good enough, that's a ticket.15:34
<cbeer>i'm sure they're good enough. but I have to serialize them into something nice.. and look up our external identifier for a path.
<ajs6f>The whole idea of the internal bus is that it should be _easier_ to work with it than network protocols. :)
<cbeer>and i think i probably need lunch before feeling up to the challenge
<ajs6f>Maybe FedoraEvent needs more methods, or a helper class?
<cbeer>FedoraEvent? do we have one of those?15:35
<ajs6f>We should… I made one and thought we merged it, quite a while ago.
Yeah, we do, but it just wraps a JCR Event for now, because I didn't know what else to do.
Maybe we now do. :_)15:36
<cbeer>ok, i'll take a look for it after lunch
<ajs6f>It might be the best place to add some convenience for this kind of stuff, or maybe FedoraObject/Datastream are.
Actually, maybe we need a FedoraResource abstraction for things like "getExternalIdentifier()" or "getURL()". It maybe should be a mixin, but I guess it would have to be an abstract superclass.15:37
* kaarefc joins15:38
<ajs6f>This also looks kind of cool:
http://incubator.apache.org/graffito/jcr-mapping/index.html
Hm, "There are no releases yet. See the menu "Getting Started" for instructions on how to download and build the Graffito JCR mapping sources." Maybe not.15:39
This:15:41
http://jackrabbit.apache.org/object-content-mapping.html
looks a lot more stable...
* travis-ci joins15:43
[travis-ci] futures/fcrepo4#215 (master - 9e7976c : Chris Beer): The build has errored.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/7951f9019856...9e7976cfeb34
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/5446815
* travis-ci leaves
* ajs6f leaves15:50
* nbanks leaves15:52
* ajs6f joins15:53
* ajs6f leaves15:54
* travis-ci joins15:58
[travis-ci] futures/fcrepo4#217 (master - 173bd68 : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/9e7976cfeb34...173bd6814837
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/5449414
* travis-ci leaves
* kaarefc leaves16:01
* VincentNG leaves16:10
* nbanks joins16:18
<cbeer>ok, found FedoraEvent16:45
hm. maybe we just need a http api method that can take the jcr:uuid or whatever and figure out where to actually route it16:46
or bring on the info:fedora uris16:47
<eddies>bite your tongue16:56
* BHSPiMonkey leaves17:00
* BHSPiMonkey joins
* github-ff joins17:10
[fcrepo4] cbeer pushed 1 new commit to webhooks: http://git.io/XvfV-g
fcrepo4/webhooks 66aaaeb Chris Beer: add test to prove we registered a webhook succesfully
* github-ff leaves
* kaarefc joins17:12
<cbeer>asynchronous messaging is hard.17:26
* kaarefc leaves17:33
* github-ff joins17:40
[fcrepo4] cbeer pushed 1 new commit to webhooks: http://git.io/P0ZFQw
fcrepo4/webhooks 24157f1 Chris Beer: prove we're receiving webhook callbacks; serialize the body as Legacy JMS ATOM.
* github-ff leaves
* github-ff joins17:41
[fcrepo4] cbeer opened pull request #37: Webhooks (master...webhooks) http://git.io/odyLjQ
* github-ff leaves
<cbeer>sorry eddies. i've cheated again and pulled tickets from the backlog that sounded more fun than stuff in the iteration :P17:42
* github-ff joins18:01
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/YfboMQ
fcrepo4/master 0f8a3ff Chris Beer: add low-level storage service to the repo.xml for the webapp
* github-ff leaves
* nbanks leaves18:08
<cbeer>hello18:21
<pivotal-bot>Hello, cbeer
* pivotal-bot leaves
* pivotal-bot joins18:22
<cbeer>hello18:23
<pivotal-bot>Hello, cbeer
* pivotal-bot leaves18:24
* pivotal-bot joins
* pivotal-bot leaves18:25
* pivotal-bot joins18:26
Chris Beer edited "fcrepo 4 events feed should be sorted reverse-chronological." https://www.pivotaltracker.com/story/show/45512057
Chris Beer delivered "fcrepo 4 events feed should be sorted reverse-chronological." https://www.pivotaltracker.com/story/show/45512057
<cbeer>there.
* travis-ci joins18:32
[travis-ci] futures/fcrepo4#220 (master - 0f8a3ff : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/173bd6814837...0f8a3ff83bbe
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/5454841
* travis-ci leaves
* nbanks joins18:37
* barmintor leaves18:43
* nbanks leaves18:47
* VincentNG joins18:50
* VincentNG leaves18:51
* nbanks joins19:16
* nbanks leaves19:29
* nbanks joins19:57
* nbanks leaves20:08
<pivotal-bot>Chris Beer edited "Extend the RepositoryService content retrieval to support clustering." https://www.pivotaltracker.com/story/show/4599635720:31
* nbanks joins20:37
* nbanks leaves20:50
* nbanks joins21:18
* nbanks leaves21:29
* nbanks joins21:59
* nbanks leaves22:11
* nbanks joins22:39
* eddies leaves22:49
* nbanks leaves22:52
* nbanks joins23:20
* nbanks leaves23:32
* eddies joins23:35
* eddies leaves
* eddies joins
* nbanks joins00:01
<cbeer>authn--00:07
* github-ff joins00:08
[fcrepo4] cbeer created broken-authn (+1 new commit): http://git.io/Bmo5BQ
fcrepo4/broken-authn e1dafc8 Chris Beer: some authn stuff that doesn't wok
* github-ff leaves
* nbanks leaves00:14
* nbanks joins00:42
* nbanks leaves00:55
<pivotal-bot>Edwin Shin added "Repository Reporting and Metrics" https://www.pivotaltracker.com/story/show/4608787901:18
Edwin Shin added "Deployment" https://www.pivotaltracker.com/story/show/4608793901:19
Edwin Shin edited "Reporting and Metrics" https://www.pivotaltracker.com/story/show/4608787901:20
Edwin Shin edited "Document the use of an indexer template" https://www.pivotaltracker.com/story/show/4409311101:22
Edwin Shin edited "An OAI-PMH provider" https://www.pivotaltracker.com/story/show/43944081
Edwin Shin edited "Document the use of an indexer template" https://www.pivotaltracker.com/story/show/4409311101:23
Edwin Shin edited "An OAI-PMH provider" https://www.pivotaltracker.com/story/show/43944081
Edwin Shin edited "Create ActiveFedora models that mimic GSearch indexing output" https://www.pivotaltracker.com/story/show/45395527
Edwin Shin edited "Verify ease of deployment of fcrepo-kitchen-sink to AWS" https://www.pivotaltracker.com/story/show/4569231701:24
Edwin Shin edited "fcrepo 4 events feed should be sorted reverse-chronological." https://www.pivotaltracker.com/story/show/45512057
Edwin Shin edited "Support different checksum algorithms beyond sha1" https://www.pivotaltracker.com/story/show/4570968301:26
Edwin Shin edited "Write documentation for running fcrepo 4.x as a WAR under e.g. tomcat" https://www.pivotaltracker.com/story/show/45507631
Edwin Shin edited "Reporting and Metrics" https://www.pivotaltracker.com/story/show/46087879
Edwin Shin edited "Deployment" https://www.pivotaltracker.com/story/show/46087939
* nbanks joins
<pivotal-bot>Edwin Shin edited "Write documentation for running fcrepo 4.x as a WAR under e.g. tomcat" https://www.pivotaltracker.com/story/show/45507631
Edwin Shin edited "Expose object audit logging through a REST API" https://www.pivotaltracker.com/story/show/4394268701:28
Edwin Shin added "Report on objects by mimetype" https://www.pivotaltracker.com/story/show/4608840301:31
Edwin Shin edited "Report on objects by mimetype" https://www.pivotaltracker.com/story/show/46088403
Edwin Shin added "Report on objects by filesize" https://www.pivotaltracker.com/story/show/4608846901:34
Edwin Shin added "Report on ingest rate" https://www.pivotaltracker.com/story/show/4608852301:36
* nbanks leaves
<pivotal-bot>Edwin Shin added "Report on object access" https://www.pivotaltracker.com/story/show/4608856901:38
Edwin Shin added "Install Fedora via apt-get" https://www.pivotaltracker.com/story/show/4608864501:41
Edwin Shin edited "Install Fedora via apt-get" https://www.pivotaltracker.com/story/show/46088645
Edwin Shin added "Programmatic configuration of Fedora" https://www.pivotaltracker.com/story/show/4608877101:46
Edwin Shin edited "Programmatic configuration of Fedora" https://www.pivotaltracker.com/story/show/46088771
* nbanks joins02:04
* kaarefc joins02:15
* kaarefc leaves
* eddies leaves02:47
* eddies joins
* eddies leaves
* eddies joins

Generated by Sualtam