Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* github-ff joins00:40
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/Gc_REQ
ff-modeshape-prototype/master c903992 Chris Beer: more fcrepo 3.x compatibility. hydra uses /history not /version
* github-ff leaves
* eddies leaves01:39
* eddies joins01:40
* eddies leaves
* eddies joins
* eddies leaves02:35
* eddies joins02:39
* eddies leaves
* eddies joins
* eddies leaves04:07
* fasseg joins04:47
hola...although probably just idlers and sleepers in here ;)
* Guest71006 joins05:11
* Guest71006 leaves
* Guest71006 joins
<fasseg>hola guest eddie05:42
have you guys seen this: http://www.bibalex.org/isis/UploadedFiles/Publications/DAR_%20A%20modern%20institutional%20repository%20with%20a%20scalability%20twist_1.pdf ?05:43
matthias hahn dug that up....the library of alexandria did some things to their data model and fedora (married with 4store e.g.) for better scabality...05:44
I'll just post that to ff-internal
<Guest71006>fasseg, had not seen this05:50
eww, brb. didn't realize i was logged in as guest05:51
<fasseg>"/nick eddies"
<eddies>surprised to see something coming from alexandria folks
they dropped off the radar for awhile after the uprisings05:52
<fasseg>hmm can't see a date on the paper but their references are from 2011 so it can't be that old
I think I missed them completely05:53
never even heard of them using fedora
<eddies>well they were talking about adopting hydra at one point
so definitely fedora
<fasseg>I like the idea of letting the users know that a good content model can benefit performance quite a bit05:55
and maybe an example of how to reduce triples in fedora using a "bad" and a "good" example05:56
<eddies>i'm not seeing a mention of how many triples they're dealing with
<fasseg>hmm lemme see..05:57
<eddies>but only 450,000 objects, so i'd guess not more than tens of millions
total disk storage is a respectful 30TB05:58
but i wouldn't expect to see scale issues with half mil objects or even a hundred million triples05:59
<fasseg>oh so there's only 1 4store node06:00
I thought they were scaling out 4store in the env
and those books are probably N images each06:01
so a lot of datastreams again
but just referencing them...06:02
but i guess the implemented trippi for 4store06:03
so they're using fedora for metadata only...like DOMS06:05
<eddies>what's DOMS?06:22
<fasseg>Asger's repo at the SB
<eddies>hey, are there any breweries near where you live?06:24
or, essentially, any special german beers you could pack for your trip?
<fasseg>sure...there's breweries all over the place in germany
<eddies>would be nice to have a bottle or two we could raffle as prizes06:25
and if it's something that folks can't find in the u.s., even better
<fasseg>hmm, lemme see.... there's famous flensburger although it's not from my area, but it comes in a stylish bottle
<eddies>i don't know my beers…hold on let me look that one up
and this would be our beer: http://en.wikipedia.org/wiki/Rothaus06:27
from the region i'm from
the black forest namely
<eddies>i'm trying to find highly rated beers on beeradvocate: http://beeradvocate.com/beer/profile/2854/671006:29
<fasseg>One question btw: I have a friend in Chicago with whom I went to university, would it be alright to have him over at the app for a beer?06:30
<eddies>this one looks slightly better reviewed: http://beeradvocate.com/beer/profile/2854/6711/?sort=serv&servT=1&start=0
yeah, but i'd say in the evening06:31
<fasseg>that's a starbier though
<eddies>what's a starkbier?
more O/oo basically
<eddies>ah, here's their top beers for germany: http://beeradvocate.com/lists/de06:32
<fasseg>1st one is hefe...
that's quite a different taste then pils
I'd recommend sticking to pils or export06:33
<eddies>i'm guessing anything with a lot of reviews means its available in the u.s.
<fasseg>and it's all 5 O/oo+
they're just rating alcohol content
not taste ;)
well, i leave it to you. but if you can get a couple bottles to bring, that would be great06:34
<fasseg>well tanennz�pfle is at 4.11 with 81 reviews
and that's the rothaus thingie
yeah leave it to me.....
btw did i mention that i dont drink06:35
well not never but not really a beer man
<eddies>i didn't know that
<fasseg>heh no problem I *think* I can still party
but i haven't had more than 2 glasses of anything since about 10 years ;)06:36
<eddies>if you can bring two bottles (i'd say different if possible), that would be great. if you let me know the names (or just send me links from beeradvocate), i can include it on the list of raffle prizes
<fasseg>yeah sure, I'll have to go get groceries anyway later today...06:37
i'm probably going to bring a bottle of scotch from the duty-free since there really isn't any special bring to bring from singapore06:38
*special beer06:39
i think jason said he doesn't drink either
<fasseg>http://www.quickmeme.com/meme/35cmhu/ ?06:40
all of reddit is full of candian apology memes atm...06:41
and what about having an old firend from university times over for one evening in chicago?06:43
would that be acceptable....? although he *is* a lawyer06:44
and a sports angler.....jesus06:46
well i guess you need two stereotypes to make a clich�e06:47
<eddies>yah. i said above that should be fine, but it should be in the evening06:48
which i assume if he's working is going to be the case anyway
<fasseg>oh didn't see that sry06:50
yeah just an evening....
cool thx
<- says the guy chtting away on IRC...
* eddies leaves08:00
* eddies joins08:12
* eddies leaves
* eddies joins
* jonathangee joins08:20
* JasonDGI joins08:25
* jonathangee leaves08:29
<JasonDGI>whats the command to tunnel into port 80 of futures6?08:42
* jonathangee joins08:43
* github-ff joins09:18
[ff-modeshape-prototype] ajs6f pushed 1 new commit to master: http://git.io/Td_i4A
ff-modeshape-prototype/master 7d14804 ajs6f: Cleaning unused imports and changing variable to field for efficiency
* github-ff leaves
<JasonDGI>just got a notice, futures6 is out of disk space09:32
<eddies>jasondgi: can you email anusha & cc ff-internal09:34
<eddies>i thought port 80 was opened
along w/ a number of others
<JasonDGI>maybe it is, ill check again09:35
* ajs6f joins09:38
Chris— are you working on the JMS part of that event listener, or would you like me to put it into place?09:39
<cbeer>ajs6f: around now. go ahead if you want. if i haven't clicked start in pivotal, there's a great chance i haven't done anything to it10:19
<ajs6f>cbeer: by Pivotal, we mean:
<cbeer>ajs6f: yep10:22
<ajs6f>Cool. I'mm all over it like brown on rice.
<cbeer>JasonDGI: i'll find space on futures6. (although, i'm not sure where it all went. huh. )10:23
* barmintor joins
<JasonDGI>i noticed it was causing passenger to fail as well
<cbeer>and, yes, port 80 is closed. only 5678 is open.10:24
i asked for 80 to be opened, but socks proxies work fine. i can send something to ff-internal
eddies: it'd be super nice to have a dns we could work with faster than what we have now.. any chance duraspace would get us access to e.g. fcrepo.org?10:25
<eddies>i can certainly ask
<cbeer>oh, and i've finally gotten through my email, so i guess port 80 should be open now, huh?
JasonDGI: ok, we have space again on futures6. fedora was full of data10:27
and it looks like they bumped diskspace to 60G
let me move fedora somewhere where i'll look for it next time10:28
<ajs6f>cbeer: before I do that, though, it looks like our ActveMQ config is causing some problems for jetty:run, so I want to fix that.10:29
<cbeer>yep. i saw that, commented it out, and forgot to tell you.
<ajs6f>Well, I'm going to refactor how we load the Spring contexts so that we have one master context. That should keep us a little more sane, until we get to some modularized world.10:32
<cbeer>JasonDGI: looks like there's more fedora configs i need to update.10:33
* github-ff joins10:38
[ff-modeshape-prototype] ajs6f pushed 1 new commit to master: http://git.io/DeH0hQ
ff-modeshape-prototype/master b520319 ajs6f: refactoring Spring to make jetty:run work again
* github-ff leaves
<JasonDGI>is the WGBH web frontend on futures6 hydra?10:40
until we get a better name for it, http://ff.cbeer.info/
and i think i left it pointing at modeshape
and found a couple things to change in ff-modeshape-prototype10:42
<JasonDGI>is passenger+hydra taking over the web frontend entire? i cant access drupal+islandora - the url resolves to wgbh10:43
<cbeer>JasonDGI: it's all virtual-host based. i think i made up hydra.fcrepo.org and islandora.fcrepo.org as the vhost names10:44
ajs6f: they're using the dsMIME part of the datastream profile to.. do something.
and we're just sending an empty string
<ajs6f>cool. we've actually _got_ that. we just need to put it in the response.10:45
<cbeer>yeah. i was about to do it, and then i got confused by <#-- -->
<ajs6f>oh, foo.
Let me explain that:
<cbeer>thanks. the freemarker docs are scarying me10:46
<ajs6f>we were leaving live objects (meaing still conected thru a session to the repo) in the stuff for the templates.
when i fixed the concurrency problems, I made those objects dead, which fouled the templates.10:47
we really got to get a proper view framework in there.
so we can fix at least two ways
1) do value-copies of everuthing for the templates
<cbeer>(boo, right?)10:48
<ajs6f>2) change way the templates are renderted to happen before the ssesion is logged out
I'm agnostic.
<cbeer>2 seems more flexible for now10:49
<ajs6f>there's not much data in the templates and all of it should available as primitive datatypes.
<cbeer>we can keep doing things ad-hoc
<ajs6f>yeah, that wouldn't hard.
<cbeer>ok. i can take care of it
<ajs6f>just have et stream from freemarker and use that in the Response.
yeah, frremarker docs… bleh. especially for a templating engine!10:51
<cbeer>and.. i can (and should) mark the stream as 'final', right?10:52
<JasonDGI>I can't hit *.fcrepo.org from the outside, is that supposed to happen?10:54
<cbeer>JasonDGI: *.fcrepo.org aren't in DNS.
but i did stick something in /etc/hosts on futures6
so you can mess with your local hosts file
<JasonDGI>ok, ill do that, thanks
<cbeer>or i can add something off cbeer.info (or, you can add something off a domain you control)
<JasonDGI>hosts will be fine10:55
<ajs6f>yep to final. use final liberally, anytime you don't expect to change the identity of an entity.10:56
<cbeer>JasonDGI: and fedora is running at localhost:8080/fedora, solr multicore is at localhost:8080/solr10:58
i went looking for the islandora solr configs and failed
but if you want to point me at them, i can create a core for you
<JasonDGI>ill set that up
<cbeer>or that.
<JasonDGI>should we use separate cores? or share one?
<cbeer>separate, please.10:59
solr home is /opt/solr
ajs6f: i still get those stupid freemarker "Expression ds.getParent is undefined "11:00
maybe i'll spend quality time with the freemarker docs and see if there's something special that'd make the parser happier11:01
* anusha joins11:02
<cbeer>oh, shoot. time for the call isn't it?
<cbeer>ok, on.11:03
<eddies>jasondgi: are you joining the call?11:04
<JasonDGI>im dialling in now
<ajs6f>cbeer— is there a "session is not live" part of the stacktrace?
<JasonDGI>jon cant make it so ill wont be able to call directly
<ajs6f>that would be the same problem
<cbeer>ajs6f: nope. this is the same thing i was seeing before
<cbeer>yep. i'm reading things about JavaBeans
<ajs6f>Yeah, that's why I introduced an ObjectWrapper11:05
I thought that fixed it.
<cbeer>hm. let me find it
s/helping/doing all the work/
<cbeer>playing a giant game of messaging-passing hot potato.11:08
* github-ff joins11:10
[ff-modeshape-prototype] cbeer pushed 3 new commits to master: http://git.io/rGfOYg
ff-modeshape-prototype/master 161a523 Chris Beer: render templates before closing JCR sessions
ff-modeshape-prototype/master f942f41 Chris Beer: add our contentType to the datastream response
ff-modeshape-prototype/master 2ae5da0 Chris Beer: just pass the parent object into the templates, i'm sick of this intermittent freemarker problem
* github-ff leaves
<cbeer>s/obvious/obvious to Adam/11:11
* github-ff joins11:12
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/5muREA
ff-modeshape-prototype/master 37a8053 Chris Beer: put the FedoraRepository APIs at / with the rest of our API calls
* github-ff leaves
<cbeer>ok, that should be enough for hydra, i'll take a look after the call
<ajs6f>+1 cbeer!11:14
<ajs6f>"s/obvious/obvious to Adam/"?! What's that all about? {grin}11:15
ajs6f: you said something about spring being obvious :P11:17
<ajs6f>Obviously opaque!
just hope for more rain!11:22
which is fine, until i get Adam to do all the work :P11:23
<ajs6f>One-handed! And without a net!
* github-ff joins11:30
[ff-modeshape-prototype] ajs6f pushed 1 new commit to master: http://git.io/PUuulA
ff-modeshape-prototype/master 619e634 ajs6f: Cleaned out unused namespaces in master.xml
* github-ff leaves
<cbeer>ok, the hydra head works as expected now that the mimeType is being passed11:33
<JasonDGI>did you get it entirely wrking?11:35
* github-ff joins
[ff-modeshape-prototype] ajs6f created ActiveMQTopicPublishing from master (+0 new commits): http://git.io/UYK9Qg
* github-ff leaves
<anusha>Are any of you running something on 8080 on futures6?
<JasonDGI>i think tomcat+solr is11:36
<cbeer>eddies: maybe we should use fedora4lib.org as our host name for all our work? maybe that'd be easier than getting duraspace on board?11:38
k, and now i've removed the WGBH branding from that site11:41
i didn't know we had faculty and staff on this project
<eddies>cbeer: if you don't mind be being the bottleneck then for dns changes
i just registered that personally11:42
<ajs6f>"Professor Beer"?
<eddies>*mind me being the bottleneck, i meant
<cbeer>eddies: that's fine with me. i suspect you just doing it is faster than asking you to ask duraspace
just let me know what A records or CNAMEs to add11:43
<cbeer>then, can you CNAME irclogs to gluck.cul.columbia.edu; hydra, islandora and modeshape to futures6.bodleian.ox.ac.uk?
(i'm finally getting good at spelling bodleian.)
<anusha>I'm restarting tomcat. Fedora is not responding11:49
<cbeer>eddies: then i guess it was silly to ask in #duraspace :P
<eddies>cbeer: added cname records for (irclogs|hydra|islandora|modeshape).fedora4lib.org11:54
<ajs6f>Urg. All of the ActiveMQ docs and Spring XSD files are borked. Gone from the Web. How cool is that?
<eddies>oi. folks are already registering…did someone go public w/ our link?11:55
<ajs6f>Is it one of us?
(I didn't even know we had the link yet.)
<eddies>david lacy from villanova and becky yoose from grinnel11:56
<ajs6f>Well, they're cool.
<eddies>yeah, but how?!
<ajs6f>At least you'll have someone to smoke cigarettes with.
HOnestly, I don't know.
<eddies>cbeer: ack. i think our intellij license expired11:57
<ajs6f>I know that the Fedora YourKit license no longer does upgrades. I've been meaning to ask DuraSpace about that.11:58
<eddies>seriously, a third registration. somewhere the word is out :P
well, i'll forward the license, but i think this won't work anymore :-(11:59
ajs6f: i'd have to make new inquiries to jetbrains and yourkit for renewals12:01
<ajs6f>The old version did okay for Chris and myself when we had a memory leak. Just something to put in the hopper.12:02
<anusha>cbeer: Could I ask for your help Please. Looks like tomcat no longer recognizes fedora in futures6. Did you install tomcat using apt-get or are using the inbuilt one at /opt/fedora
* anusha leaves12:10
<eddies>aha. here's where the registrations are coming from: http://duraspace.org/fedora4lib-code4lib12:11
How dare they eagerly swarm us like the Beatles getting off a plane!
<eddies>i should go plug in some google analytics
well, i'm going to have to cut off registrations at some point12:12
we probably can't fit more than 25ppl/night
<ajs6f>Ah, the price of fame.
<eddies>well, and they'll drink all our beer!12:13
<eddies>think of the beer^H^H^H^Hchildren!
who is avessey?
<eddies>in our irc channel since yesterday
or so
<avessey>Iunno. :P
I'm a dev at DGI... Just hangin' out.
<eddies>not that i mind. just curious =)12:15
<eddies>ah ok. hi there!
<JasonDGI>how do i kick someone :D
<cbeer>eddies: can you re-up the Intellij license? it apparently expired in 2012212:21
<eddies>scrollback in your buffer
that's unfortunately the only license we have12:22
i'll have to ask jetbrains for a new license
ok, http://irclogs.fedora4lib.org/ vhost in place.12:28
why can't they agree what to call apache httpd?12:30
<eddies>The best day on irc: http://irclogs.fedora4lib.org/2012-12-27.html
<cbeer>ok, hydra and islandora vhosts in place.
<eddies>thanks chris. cbeer++12:31
we don't really have karma tracking in this channel, eh?
<JasonDGI>i was just about to ask about that
<cbeer>nope. no zoia12:32
<ajs6f>I thought the Universe took care of that for us. Kind of a container service.
<JasonDGI>dont you have to sign up for that?
<ajs6f>Like Travis, eh?12:36
<eddies>i love running rake deploy for publishing a web site
<ajs6f>I've got to pedal to the office for a meeting. I'll check in here later.
Got to stop doing that.
* ajs6f leaves
<eddies>mail went out to the code4lib list12:50
so now we're really public
* eddies afk
<JasonDGI>islandora wants apache's clean urls enabled, will that cause a problem with anyone?13:11
<eddies>i don't even know what that is13:12
but we are a dirty bunch
<cbeer>JasonDGI: you mean mod_rewrite?
<JasonDGI>converts "/drupal/?q=islandora/" => ""/drupal/islandora/"
yeah, i think it is rewrite
<cbeer>you should be able to do that in the islandora vhost
<JasonDGI>right, thanks
* ajs6f joins13:31
<cbeer>hm. the infinispan docs tell me:13:49
"It is imperative that implementations do not close the stream after finishing with it."
<ajs6f>Do they say why?
<cbeer>not that i've found13:50
tempting to just wire something up and see what happens.. but that's probably not healthy.
<ajs6f>Well, if it's quick...
<cbeer>should be, now that i've figured out what all these methods are supposed to do13:51
i'm scared by a couple of the methods... e.g. loadAll()
but, what can you do, i guess.
<ajs6f>I'm guessing those are for ephemeral caching type uses?13:52
<cbeer>i wonder if the warning has to do with transactions?
<ajs6f>I'm sure thos semantics should cross...13:56
<cbeer>looks like the transaction stuff just calls my methods
so, hm.
<ajs6f>I just commited the JMS publishing bit, with test. Some of the repository-wise methods are failing tests— does that have to do with you moving their URLs, Chris?
<cbeer>looks like it might.13:57
i see some fedora references in there
sorry about that
<ajs6f>Well, now Travis is going to throw his usual hissy fit. Little whiny baby.
No trouble. Maybe you want to merge thebranch and then fix them?
Or the other way around?
I've got a meeting, but I'll check in a little later.13:58
* github-ff joins13:59
[ff-modeshape-prototype] ajs6f pushed 1 new commit to ActiveMQTopicPublishing: http://git.io/LF_nZg
ff-modeshape-prototype/ActiveMQTopicPublishing b32ef8f ajs6f: Publish to JMS topic
* github-ff leaves
* ajs6f leaves
* github-ff joins
[ff-modeshape-prototype] ajs6f opened pull request #25: Publish to JMS topic (master...ActiveMQTopicPublishing) http://git.io/gFHHHA
* github-ff leaves
<cbeer>hm. i'm not sure we actually have enough information down in infinispan to do a good job at durable storage14:09
i guess that's why modeshape did their binary store at the modeshape level
<JasonDGI>i cant seem to get mod_rewrite working for the islandora vhost14:11
<barmintor> "I'm not sure we have enough information down in [akubra - infinispan - lily] to do a good job at durable storage"
deja vu :P
<eddies>barmintor: where did that come from?14:15
<barmintor>eddies: I was just noticing a similarity in what cbeer just said and what we've said about two other platforms
<cbeer>JasonDGI: tell me what you need (and how to know it works) and i'll take a look after lunch14:16
barmintor: i know! let's add a HintsProvider!
<barmintor>eddies: who is signing up for beer/pizza/hacking?14:17
<JasonDGI>ive always just enabled mod-rewrite and it works
http://futures6.bodleian.ox.ac.uk/drupal/#overlay=%3Fq%3Dadmin/config/search/clean-urls is the status page telling if its online
<eddies>i thought i shared the spreadsheet w/ everyone
<barmintor>cbeer: I'm sure we can pass along any relevant information as a map of strings
<JasonDGI>it may need a user/pass
<cbeer>Strings should be enough for anybody.
JasonDGI: you missed a / on your <Directory>14:18
<eddies>oh well, ben has access and a bunch of steering folks
<JasonDGI>was that it!?!?
<cbeer>JasonDGI: var/www/islandora vs /var/www/islandora
does that do it?
* eddies afk
<barmintor>eddies: I see it now
<eddies>anyone know how to embed tweets for #fedora4lib...14:21
<cbeer>JasonDGI: ok, taking a deeper look.
<eddies>pull requests welcome :P
<cbeer>man.. drupal has gone lightbox happy.
<eddies>actually, just go ahead and push it
<JasonDGI>this is the rewrite guide im using: http://drupal.org/node/1536514:23
but its never been this difficult before
<cbeer>JasonDGI: i think i've done it. drupal was being weird when i went to the futures6 url
much happier at http://islandora.fedora4lib.org14:24
<JasonDGI>cbeer: you keep replacing "helping" with "doing all the work", jeez
<cbeer>JasonDGI: so, ajs6f is doing all my work, i'm doing all your work.. i guess you should start doing anusha's work?14:25
<barmintor>Things I didn't think would be working on this week but am: SWORD deposits for our acad. repo.14:26
<cbeer>and there's actually a consumer for that service?14:29
yeah, this is exactly the same problem as Akubra.14:31
but they have cooler class names.
" ImmortalCacheEntry"
<barmintor>wow, those are awesome class names. Pay attention, Spring devs!14:32
<JasonDGI>cbeer: what did you do to fix drupal? cause now the rest of it is broken14:34
<cbeer>hm. i just lost a pivotal ticket
JasonDGI: went to http://islandora.fedora4lib.org/, found the clean urls setting, turned it on.
<JasonDGI>so it just plain doesnt like the futures6 address?14:35
<cbeer>yeah, don't use that. that's a totally different vhost.
<JasonDGI>i hate stuff
<cbeer>i guess it must be time for a modeshape/infinispan forum question..14:36
yeah, i guess we can't do this down at the infinispan layer without a lot of hassle.. which is why modeshape has a Binary Store, I guess14:39
that doesn't seem like the end of the world.
<eddies>barmintor: sword into fedora?15:02
<barmintor>eddies: yeah. Reworking that old webapp to be a JAX-RS resource
<eddies>i want an ImmortalCacheEntry
<barmintor>going to wire it directly into our FCR 3.6 install
<eddies>which old webapp do you speak of?
not that sword 1.0 webapp on sourceforge?15:03
<barmintor>eddies: you get namechecked in the commit messages, actually
<eddies>i do?
that app had 6 downloads this week!
<eddies>so i thought all the sword work was on github: https://github.com/swordapp15:05
and i'm biased, but i prefer my fork of sword2-server: https://github.com/mediashelf/sword2-server15:06
<barmintor>this is a sword 1.3 (I think) requirement15:07
<eddies>and there is a sorely incomplete implementation of sword2 for fedora: https://github.com/mediashelf/sword2-fedora
ah ok
then sourceforge it is :-P
<JasonDGI>localhost:8080/fedora is giving me a 404
and futures6…, and fedora4lib are too
<cbeer>cbeer@futures6:~$ curl "http://localhost:8080/fedora" -I15:47
HTTP/1.1 302 Moved Temporarily
oh, but.. hm.15:51
* JasonDGI leaves15:52
<cbeer>ah, looks like we had a phantom tomcat instance running15:53
* github-ff joins15:57
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/zlL3NQ
ff-modeshape-prototype/master a3373cd Chris Beer: fix FedoraRepositoryTest for new urls
* github-ff leaves
* github-ff joins15:58
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/1MzOJw
ff-modeshape-prototype/master 42973f2 Chris Beer: Merge pull request #25 from futures/ActiveMQTopicPublishing...
* github-ff leaves
* jonathangee leaves16:00
<cbeer>it's not going to be a good day if i'm cloning modeshape and infinispan.16:01
ok. time to figure out how modeshape sends stuff to infinispan in the first place.16:12
* ajs6f joins16:17
<cbeer>ajs6f: so, i've been looking at modeshape/infinispan and how to write our own cache store (that stores things durably, doesn't hold open file handles forever)16:26
and it's not pretty.
<cbeer>i'm looking at overriding org.modershape.jcr.value.binary.infinispan.InfinispanBinaryStore
which is where we might have access to the data we need to make a pleasant directory layout on disk16:27
<ajs6f>LIke what data? Metadata about the nodes (name,path, that kind of thing)?
<cbeer>barmintor had flash-backs to Akubra
<ajs6f>Give that man some Thorazine, stat!16:28
<cbeer>yep. all infinispan gets is an Object key and a *Stream
That's _less_ than Akubra.
<barmintor>it is hilariously similar to the Akubra Blobstore interface
BlobstoreConnection, that is
<ajs6f>Okay, we may have to pack the Object key.
We should be able to put whatever we want in that and act on it, right?16:29
<barmintor>It depends on how things are retrieved
<ajs6f>Right, right.
Okay, take a step back with me. What counts as a "nice" filesystem strcture?16:30
What does it have to include/be constructed out of?
<cbeer>ajs6f: i think human-readable filenames would be nice.
<ajs6f>Me, too, but let's don't get greedy.16:31
<cbeer>right now they are the java hash code, i think
I'm trying to get the size of the contraband we're trying to smuggle over the border to Infinispan as small as possible.
If we had a filesystem structure that made it possible to retrieve a resource with nothing in hand but a PID, would that be enough?16:32
<barmintor>that google doc of attendees is manually edited, right?
(is eddies in bed yet?)16:33
<ajs6f>where is that, anyway? I was wondering who signed up.
<cbeer>travis-- # keeping me honest..
<barmintor>ajs6f: you can find it through the google docs interface. Look to see what Eddie has shared with you
<ajs6f>I don't see it there. Maybe Eddie didn't share it with me16:34
<cbeer>eddies said it was with barmintor and steering group folk
not us commoners.16:35
<barmintor>Yeah! Now get back to the codemines, you lot!
So if we just use the PID for the key, that would be okay for retrieval, right? And it was get us on the road to a really nice filesystem, no?16:36
And no collions.
I'm kind of surprised that collion isn't a word. Sounds like it should be16:37
<barmintor>So far 16 people have signed up for Monday night, and 5 have signed on for Tuesday.
<cbeer>ajs6f: i don't know, what with versioning etc.
<ajs6f>Oh, versioning.16:38
<barmintor>and datastreams.
<ajs6f>To my understanding, versioning in JCR creates new nodes.
which are linked by relationships.
<barmintor>unless a FC4 pid always refers to a DS
<cbeer>oh, so by PID you might mean.. UUID?
<barmintor>yar, yar
<cbeer>i'd take a UUID.
<ajs6f>Boys, boys, there's plenty of non-colliding ID space for _everybody_.16:39
If we used PID/DSID for the key for datastreams, we'd still be okay, right?
Or would there be a problem with retrieval...?
I guess we have to come to understand how ModeShape does retrieval.
<cbeer>i'm still trying to decipher teh modeshape side of things
<ajs6f>What's available for use at that moment.16:40
Yeah, ModeShape. My ModeShape is your modeshape.
<cbeer>i think there's a SHA1 hash for binary values
i haven't figured out where non-binary values happen.
<ajs6f>Hm. "ModeShape stores all binary content by its SHA-1 hash."16:41
<cbeer>(which isn't quite true, either)
<cbeer>it's the SHA-1 hash plus a chunk identifier
and i'm happy with SHA-1, i guess16:42
for binary values
I'm surprised. I thought you wanted a more human-readable design.
<cbeer>sure, why not? they're sufficiently unique and supposedly meaningful
<ajs6f>I… guess...
<cbeer>sorry, i'd settle with SHA-!
<ajs6f>Okay with me.
<cbeer>it's these hash ids that i really dislike
<ajs6f>I've been arguing for a asynchronous durable store from the begining.16:43
<cbeer>ok, so.. i should give up on this.. and go back to my original problem
<ajs6f>Then we don't have to care as much about what MOdeShape does as long as it's fast and scalable.
<cbeer>which was all these stupid file handles
<ajs6f>But Neil hated on me.
<cbeer>i don't like the idea of copies of binary content
<barmintor>cbeer: would you settle for a slightly truncated SHA-1 hash? Because that's a type 5 UUID16:44
<ajs6f>WHy would ther have to be copies?
ModeShape just wants to find stuff where it expects it...
It's doesn't have to be there.
<cbeer>barmintor: i was thinking the jcr:uuids16:45
<ajs6f>That (the node IDs) is implementation-defined, so it's supposed to be under ModeShape control.
So Chris— with the file handles, was the problem that ISPN wants to keep the streams open? Is that why you can't just lose the handles?16:47
<cbeer>ajs6f: yes, that's the primary thing i was looking at. i guess i've strayed too far for now16:48
let me write something to their forum
asking why we're chided against closing handles
<ajs6f>Chide them right back!16:49
One thing to remember: the product _was_ built for speed, for fast high-scale caching. So they just may not have heard about this sort of use much before.16:50
<cbeer>good point.16:51
<ajs6f>So don't beat them up too badly.
<cbeer>i wonder if i should start in the modeshape forum, actually
"we're trying to do something weird with your project. tell us how crazy we are"16:52
<ajs6f>It's not what we're doing, it how we inist on doing it.
"Here, repo. You take care of this for us. But do it in _exactly_ the way we would do it. And we're humans."16:53
brb in 5
* ajs6f1 joins16:56
<cbeer>i wish the Fedora futures project main page actually said what Fedora is16:57
* ajs6f leaves
<ajs6f1>It's the future!
<cbeer>i think i'll gist this post and get our feedback before posting it. i suspect i'm going to ramble too much.16:58
<ajs6f1>Are you going right to the ISPN forum?
Remember to start with a compliment.16:59
<cbeer>i think so. unless starting with modeshape would make more sense. at least they'd be able to understand why we chose their platform
<ajs6f1>Hm. I really don't know. I'm sure it'll be all right.
<cbeer>ok, or i'll deal with a production outage.17:00
<ajs6f1>Production, smhaduction. Life is an alpha product.17:04
If you're going to follow up on the persistence thing,
I'll keep on with the JMS stream. I'mm going to close the first ticket andd start working on "Fedora Classic-style" Atom output.
* github-ff joins17:05
[ff-modeshape-prototype] ajs6f created AtomJMSOutput from master (+0 new commits): http://git.io/I3_tsQ
* github-ff leaves
<cbeer>oh, hm.17:11
I looked at ISPN-1936 again
and it says it was fixed in 5.1.3.FINAL.. and we're using 5.1.2
i think there's a 5.2 RC17:12
let me try plugging that in
<ajs6f1>Do you think that would kill the file handles thing/
<cbeer>it might. i'll give it a try
<cbeer>or 5.2.0.FINAL released 1/2517:14
but maybe not in maven yet?
or maybe i'm doing the wrong thing. more likely.17:15
<ajs6f1>What about the JBoss maven repo?
Is it there?
<cbeer>oh. Final, not FINAL
<ajs6f1>It's not _that_ final.
<cbeer>ALPHA, Beta, Final17:16
<ajs6f1>It really is that alpha.
Let's not do that in our prject.
<ajs6f1>I've worked on projects where, if we were honest, that was how our versions would have read.17:18
* github-ff joins17:21
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/gNZi5w
ff-modeshape-prototype/master 787f98a Chris Beer: remove module that shouldn't have been committed to master yet
* github-ff leaves
<cbeer>ok, and now to figure out why the tests are broken.
<ajs6f1>Failed tests: testGetNextPid(org.fcrepo.modeshape.FedoraIdentifiersTest): expected:<200> but was:<500>17:27
testGetObjects(org.fcrepo.modeshape.FedoraRepositoryTest): expected:<200> but was:<500>
* github-ff joins
[ff-modeshape-prototype] ajs6f pushed 1 new commit to AtomJMSOutput: http://git.io/5O7awA
ff-modeshape-prototype/AtomJMSOutput a70ac86 ajs6f: Changed JMS Topic to be in Atom format
* github-ff leaves
<cbeer>yeah, i was seeing that too.. but when i went to jetty:run:
org.springframework.beans.factory.BeanDefinitionStoreException: IOException parsing XML document from ServletContext resource [/WEB-INF/classes/spring/master.xml]; nested exception is java.io.FileNotFoundException: Could not open ServletContext resource [/WEB-INF/classes/spring/master.xml]
I'm not getting that, I don't think.
Let me try again.17:28
<cbeer>oh. hm. file not found.
<ajs6f1>Did we lose it from the code somehow?
I've got it in my branch and my jetty:run works.
P.S. I've stopped work on that branch because we now have Atom publishing and we have to try it against GSearch to see if I got enough.17:29
And in the right format version.
Have we got a GSearch deployed with Islandora on the Oxford nodes?17:30
You were doing that work for Jason, right, Chris?
<cbeer>that's just a drop-into-tomcat kind of thing, right?17:32
<ajs6f1>ask gert. {grin}
<cbeer>hm. and it works now.17:33
<ajs6f1>Seriously, yeah, I don't think it's terribly hard to install.
<cbeer>i give up.
<ajs6f1>You have to hook it to you repo somehow, via REST (or SOAP) and JMS.
And your indexes.
I would think that DGI would know that inside and out
It's a core piece of their architecutre.
I'm going to head home. Shoot me an email if you run into anything you want help with.17:34
o and o
* ajs6f1 leaves17:35
* github-ff joins17:42
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/yLPYRw
ff-modeshape-prototype/master 458f8e1 Chris Beer: change some routes around to make the test happy
* github-ff leaves
* jonathangee joins17:44
* github-ff joins17:46
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/HGsunQ
ff-modeshape-prototype/master 33f3905 Chris Beer: bump modeshape, infinispan and jgroups dependencies
* github-ff leaves
infinispan 5.2.0.Final seems to fix our file handles issue17:55
i love working against projects that other people maintain and fix our problems before we really know about them
deployed gsearch, but haven't configured it.18:10
* jonathangee leaves19:10
* jonathangee joins19:13
* jonathangee leaves20:56
* jonathangee joins20:59
* github-ff joins21:07
[ff-modeshape-prototype] ajs6f opened pull request #26: Changed JMS Topic to be in Atom format (master...AtomJMSOutput) http://git.io/N183sg
* github-ff leaves
* ajs6f joins
* github-ff joins21:09
[ff-modeshape-prototype] ajs6f force-pushed AtomJMSOutput from a70ac86 to 8da2557: http://git.io/I3_tsQ
ff-modeshape-prototype/AtomJMSOutput 25527bb ajs6f: Changed JMS Topic to be in Atom format
ff-modeshape-prototype/AtomJMSOutput 8da2557 ajs6f: Making tests a little clearer
* github-ff leaves
* jonathangee leaves21:30
<ajs6f>Cbeer: did you pull back the stuff you were doing with persistence?21:38
I can't find it in master...
<cbeer>shouldn't have been in master anyway
<ajs6f>Fair enough.21:39
Are we still grinding on the contract that INSPN presents?
I want to take a look at what we can do to create human readable filesystem structures.
I want to start from wherever you got to.21:40
<cbeer>go for it. now that the file handles thing seems to be fixed (still running tests), i'm suddenly much less interested in fighting that fight.
<ajs6f>Also fair enough.
<cbeer>i had some stuff in https://github.com/futures/ff-modeshape-prototype/commit/a3373cdb7e30b6ed0998f56ae5921116a7cc5842
<ajs6f>I've been (in case I haven't said it Long.MAX_VALUE times already) of the opinion that the whole argument doesn't make sense.21:41
<cbeer>not very exciting :P
<ajs6f>No. but we've got to get it done.
I'll put on my Neil&Thorny-shaped glasses, and see what I see.
i'll work on my question to the ISPN forum tomorrow
<ajs6f>Which is about what we can and can't do in terms of keys for storage?
<cbeer>worst case, the modeshape binary store (SHA-1s) isn't the worst thing in the world imo
<cbeer>yep, that.
<ajs6f>And they might very well be amenable to a patch...
<cbeer>i saw some threads going the other way too.. fewer files, b-tree stores, etc.21:43
<ajs6f>which allows for pluggable location-creation.
yeah, yeah— we've got options.
Although the fewer files one… that weirds me out.
<ajs6f>Well, let me see if I can find a way to shoehorn us into what they've already got for non-binary storage. That would be something.21:44
* jonathangee joins
<ajs6f>I'll report back soon.
<cbeer>i'll follow up on the gsearch thing, iguess
<ajs6f>Yeah, it would be cool to say we did it.
And @jonathangee would like it so much that he would buy us all the beer we could drink.21:45
<jonathangee>exotic pei beers21:46
<ajs6f>Cod beer!
<jonathangee>needs more cod21:47
<ajs6f>I honestly love cod so much… I would totally try a cod beer.
No joking.
<eddies>good morning gang =)21:48
<ajs6f>What Singaporean street food are you going to have for breakfast21:49
<cbeer>we should start having twice-daily standups!
<eddies>i think i'm just going to have a coffee
<ajs6f>The Breakfast of Champions!
Especially good accompanied by a fine cigarette.
<eddies>cbeer, only twice?
<ajs6f>You need some time apart. "And the oak tree and the cypress grow not in each other's shadow." - Khalil Gibran21:51
* jonathangee leaves21:53
<ajs6f>Okay, so it looks like this:21:55
is the structure with which as key ModeShape is saving node-affecting changes to repos.
And that fellow looks like he's got everything we want: <ol>21:56
* <li>A multi-character key uniquely identifying the repository's storage source in which this node appears;</li>
* <li>A multi-character key uniquely identifying the workspace in which the node appears; and</li>
* <li>A multi-character key representing the JCR identifier of a node, which is usually a UUID.</li>
* </ol>
What more could we ask for?
Now if ISPAN isn't using all that info, we just got to make it do so
Now the question is… what _is_ ISPAN using...?21:58
Urg. Their API is too well-designed! It doesn't offer me leaks through which I can find out what actually fulfilling their persistence contracts. Easily.21:59
Time for some exercise with Eclipse's Java Search dialog box.22:00
On a completely different note, interesting to see that they chose the same pattern for registering change as did we:22:01
looks like a kissing cousin of Google's EventBus.
Ah! in org.modeshape.jcr.cache.document.WorkspaceCache
there it is:
line 183
if (nodesByKey instanceof BasicCache && cacheTtlSeconds != null) {
node = ((BasicCache<NodeKey, CachedNode>)nodesByKey).putIfAbsent(key,
and so on.
They are using the NodeKey structure to store the nodes.22:06
* jonathangee joins
<ajs6f>So we _should_ be able to store by node name.
<cbeer>good find. continuing to do all my work :)22:07
so, i don't think modeshape 3.1 and infinispan 5.2 play well together
<cbeer>i'm getting bad transaction errors and timeouts
<cbeer>i'll dig into it tomorrow
<ajs6f>It's a minor version.
yeah, we've got six months. {grin}22:08
five months.
Not counting drinking time
'Speaking of which, Eddie?
<cbeer>http://planet.jboss.org/post/infinispan_5_2_and_modeshape_3_2 tells us to give it a try
<ajs6f>Got those bottles for you.
<cbeer>so.. maybe i'm wrong. i'll try plain modeshape and infinispan, see if we're doing something bad.22:09
* jonathangee leaves
<ajs6f>Wow! JBoss has finaly acheived enough mass to form its own planet!
Let me know if I can help.
This would allow us to schedule the creation of a durable store on our own time, not immediately.
(Using 5., that is.)22:10
Eddie— two bottles of http://www.totalwine.com/eng/product/potters-oak-barrel-reserve-cider/130951750http://www.totalwine.com/eng/product/potters-oak-barrel-reserve-cider/130951750http://www.totalwine.com/eng/product/potters-oak-barrel-reserve-cider/130951750http://www.totalwine.com/eng/product/potters-oak-barrel-reserve-cider/130951750http://www.totalwine.com/eng/product/potters-oak-barrel-reserve-cider/13095175022:11
Okay, that didn't work.
Potter's Reserve Oak Barrel Cider.22:12
It's really the old Johnny-Jump-Up.
9.5% abv. Smooth as can be. Not too sweet, not too dry.
It's what powered the Revolutionary Army.
No offense to Oxford.22:13
<ajs6f>Well, a long day has left me feeling like the inside of a barrel of Potter's. So I'll pick up this non-binary storage thread tomorrow. Have a good [night|day|morning] everyone...22:28
o & o
* ajs6f leaves22:29
* jonathangee joins22:51
* jonathangee leaves22:53