Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* jonathangee leaves00:11
* jonathangee joins09:27
* ajs6f joins10:05
<cbeer>ajs6f: for the akubra/glacier stuff, does it make sense to configure the whole AWS client from spring and pass that to the blob store connection?11:21
s/AWS client/AWS client as a bean/?11:23
<ajs6f>If you know you're only going to put on client in play, that would make a lot of sense to me. There's one thing...11:25
Watch the constructors and mutators— if you inherit from AbstractBlobStore,11:26
you're likely to make it possible to misconfigure in Spring so that no client gets injected.
That's okay by me— just a mtter of documetnation.
That should have been "put _one_ client into play"11:27
<cbeer>for prototype-level software, i'm accepting all sorts of misconfigurations, hard-coded assumptions, and my own awful java
Then, yeah, I'd inject it.
<cbeer>i'm just happy it seems to work so far. autocomplete to the rescue
<ajs6f>The real Akubra-specific logic is not about configuratiing AWS clients, anyway.
Akubra is actually a nice clean design for what it does.11:28
<cbeer>i'm able to round-trip data through my input and output streams, just need to hook up the akubra piece
and pray that fedora is smart enough to avoid creating duplicate objects.. and that your stored data doesn't exceed 1000 items (uh oh.)
<ajs6f>Great— just remember, Fedora expects the streams to arrive more or less instantly. We'll have to do some work to change that.
Duplicate objects?
Oh, I wouldn't worry about the 1000 items thing.11:29
<cbeer>sorry, blobs with the same blob ID
<ajs6f>Channelling Eddie: that's a task for the integratot.
Oh, Fedora won't do that.
<ajs6f>integrator, is what I meant.
And with p-drven storage, we should be able to deploy mutliple Glacier vaults.
If you've allowed for the vault-name to be offered in SPring config.11:30
It _is_ 1000 per vault, right?
<cbeer>it's 1000 per vault, before you need to start paginating responses11:31
and that seemed like too much work :P
i suspect we need some kind of medium-level AWS client that wraps all that low-level stuff AWS gives us
<ajs6f>Agreed. That would probably necesitate a stateful Akubra, which is icky.
Hm. Youmean to hide pagination and so forth?11:32
<cbeer>maybe. i'm curious how fast those inventory jobs are, in practice. maybe they're practically instantaneous, so it doesn't matter how inefficient the fetching code actually is
yes. you have to do something nasty to parse the json response and send back tokens each time
<ajs6f>Well, there's how fast they seem to return, and there's what Amazon actully guarantees...
<cbeer>4 hours!11:33
And the inventories are as much as 24 hrs out of date, right?
<cbeer>something like that.
so akubra-glacier will just seem flaky to start. good enough for now11:34
<ajs6f>I'm pretty sure Eddie would say, "Not in this sprint, baby."
<cbeer>and makes a demo :P
<ajs6f>Stop-motion demo.
<cbeer>how much work would it be to teach the fedora rest api about multipart uploads? after playing with them in glacier, they seem surprisingly nice
<ajs6f>I'd have to look at CXF. Let me go do that real quick.11:35
<cbeer>(i guess it's highly blob-store dependent.. staging multiparts to a write-once data store seems bad)
<ajs6f>You mean like multi-datastream ingest kind of deal? I11:36
<cbeer>no, like the aws upload.. force the client to pre-chunk an upload and do it in pieces (and/or in parallel)11:37
<ajs6f>Wait, we're using Jersey for REST. I forgot we hadn't switched that to CXF.
Oh, segmenting...
That's interesting...11:38
Looks like Jersey does support multipart. That _might_ be something we could safely overlay onto modifyDatastreamByValue...11:39
AKA POST to objects/{pid}/datastreams/{dsid}11:40
<cbeer>what is a tree hash anyway? annoying thing to try to look up (did you mean hash tree?)11:57
<ajs6f>Are you talking to me, Chris? Where did I say "tree hash"?11:59
<cbeer>i'm just musing.
<ajs6f>Ah, IRCing out loud.
* ajs6f1 joins14:47
* ajs6f leaves14:48
* ajs6f1 leaves14:51
* jonathangee leaves16:22
* jonathangee joins21:07
* jonathangee leaves21:11