Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* eddies leaves00:37
* eddies joins00:58
* eddies leaves
* eddies joins
* eddies1 joins01:33
* eddies1 leaves
* eddies1 joins
* eddies leaves
* fasseg joins06:14
* JasonDGI joins07:13
<fasseg>im off for a bit....groceries09:03
<cbeer>i continue to be annoyed at our bifurcated wiki space.10:47
<fasseg>sleep well: http://www.youtube.com/course?list=EC34B589BE3014EAEB11:02
<barmintor>sorry, booted by skype. dialed back in with gchat11:06
<cbeer>barmintor: we can talk about this later, but let me know if there's anything we should do to make a pretty demo?11:08
<fasseg>started a script documentation there: https://wiki.duraspace.org/display/FF/FF+Candidate+Test+Harness11:10
<barmintor>cbeer: it's a good question. I guess if we elaborate some datastream endpoints that allow job status, that would be incontrevertible evidence that *something* is going on in Glacier11:15
<barmintor>I am not authorized to look at the configuration docs: http://docs.jboss.org/modeshape/latest/manuals/reference/html/configuration.html#clustering11:25
<cbeer>yeah, i'm trying to find that page too :P11:28
i think jboss is broken11:29
jboss.org is down, at least
oh. EC2. i ought to use that rather than bugging sysadmins.11:33
Infinispan's CassandraCacheStore leverages Apache Cassandra's distributed database architecture to provide a virtually unlimited, horizontally scalable persistent store for Infinispan's caches.
(infinispan is the modeshape persistence layer)
you know, because that's how tickets works.11:36
jonathangee: mvn clean jetty:run11:37
(maven 3, i think)
* github-ff joins11:39
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/xQBINg
ff-modeshape-prototype/master 5c0b34f Chris Beer: Update README.md
* github-ff leaves
<cbeer>anyone else having trouble hearing eddies?
<eddies1>let me reconnect11:40
frank, you will drop
<barmintor>you have to say more than 3 words for us to test :P
<cbeer>ok, looks like i'll need to look at:
<barmintor>#4dora <-- winner, imo
<JasonDGI>4dora the explorer
<cbeer>eddies1: just use one of the github pages themes? http://pages.github.com/
* jonathangee leaves12:05
* jonathangee joins
* jonathangee leaves
* jonathangee joins
<fasseg>some doc about the random test data set: https://wiki.duraspace.org/display/FF/FF+Candidate+Test+Harness12:08
<cbeer>eddies1: ok, i'm going to figure out how to launch some EC2 instances and store modeshape content into S3 to start12:09
<fasseg>hmm maybe the name "random" gives the wrong impression
<cbeer>i'm having trouble wrapping my head around the whole inifispan clustering, but i think it's just me.
<fasseg>called it "generated"
<cbeer>(boom, duracloud.)12:10
<eddies1>cbeer: if you're not getting early traction/success against modeshape + s3, i'd just bring up ebs volumes12:13
<cbeer>eddies1: supposedly infinispan ships with jcloud integration12:14
<eddies1>i don't know what jcloud is, but it must be shiny since it has cloud in the name ;-)12:15
<cbeer>cloud provider abstraction layer
<fasseg>and a lowercase j
<eddies1>oh someone did that eh? i remember the duracloud folks were talking about that being a missing piece in the market and doing it themselves12:16
<cbeer>and then refused to pivot onto it. i think they came out pretty close to each other12:17
<fasseg>hm based on the name i though this was another javascript backend
<eddies1>we should be different and do all caps. FEDORA FUTURES, BE LOUD AND PROUD
<cbeer>but what do i know
<fasseg>feastora for the fedor happening?
<cbeer>cool, an aws api client in eclipse
<barmintor>#d4a = defoura = fedora
<JasonDGI>+1 feastora12:19
<barmintor>"Developer Blobstore"12:21
<fasseg>invite to the fedormatorium12:22
<eddies1>ff: FedoraFest
fedoratorium, i think
<eddies1>two developers enter, one developer leaves12:23
<eddies1>we don't need another repo… dum dum dum
<eddies1>oh wait, that's FedoraDome. i was just too eager to go mad max12:24
<barmintor>Fedoradrome is a story about an underground repository architecture whose APIs turn everyone who uses them into blobstores.12:25
<eddies1>i'm sorry, i can't hear anything now except for tina turner
i want tina turner to come to our house now
<cbeer>hm. you can't launch spot instances from the eclipse api :/12:26
<eddies1>i've only ever launched/managed instances from the command line or using the aws web interface12:27
<cbeer>yeah, i've only done it from the web ui
<fasseg>"the fedora den" has a nice ring to it12:43
<barmintor>If we just photoshop the top of the X to look like a 4: http://images2.fanpop.com/images/photos/7600000/Movie-Poster-the-x-files-fight-the-future-7686985-1000-1254.jpg12:46
FEDORA $: Fight the Futures
<barmintor>what is wrong with this mock!12:48
<eddies1>barmintor: oooOOOOooo
<barmintor>FEDOR4 isn't as short as 4DORA, but it looks cooler12:50
<fasseg>Looks a bit sms-y12:52
zomg! meet up at4dora. yolo!
<barmintor>If you think I won't tweet about this with a #YOLO tg, you've got another thing coming12:53
<barmintor>Maybe that's the approach we should take: "You only ingest once". "You only deposit once". "Never say never again"12:55
<fasseg>hehehe literally: lol
<cbeer>i'm not smart enough to use the ec2 command line tools12:59
guess i might need to spin up a real instance rather than a spot request
<JasonDGI>does lily have a web frontend?13:02
<fasseg>For records and stuff?13:03
<fasseg>I just used the rest API and postman extension
err sry i wasnt supposed to post that link...wtf?13:04
<fasseg>but theres the HDFS frontend at 50060 or sth which lets you browse the ingested binaries
the solr index is also exposed at the tomcat port13:05
so solr at localhost:898313:06
<JasonDGI>ok, i tried that port with no luck, ill try again13:07
and just checking hdfs.....mom13:08
<JasonDGI>will it be blocked if im not using localhost?
<fasseg>hmm dunno
<JasonDGI>kk, ill let you know what happens
<fasseg>do you get a HTTp response?
<JasonDGI>404 on tomcat, unable to connect on every other port13:09
<fasseg>I have these:13:11
HDFS: http://localhost.localdomain:48999
HBASE: http://localhost:60010
(records are in HBASE)
<JasonDGI>ok booting up lily now13:12
<fasseg>solr admin: http://localhost:8983/solr/admin/
and solr is here: http://localhost:8983/solr
i can reach them all
<cbeer>eddies1: can you see the AWS instances we create? i think i need some help
<eddies1>cbeer: sec. i'll check13:13
<JasonDGI>ok yeah its working now
<cbeer>eddies1: i've tried to launch some instances, but they're immediately terminated. and, as far as i can tell, i don't have any way of finding out why.
<fasseg>it also tells you about the URLs in the lily logs..
<eddies1>i see 5 terminated instances
<cbeer>yes. so, looking at i-195b696813:14
any idea what's wrong?
<eddies1>State Transition Reason: Client.InvalidManifest: Invalid image manifest ami-6d842b04
bad ami?
<cbeer>k, i'll look again. thought that was the official amazon image13:15
<eddies1>yeah, same msg for all 5
<cbeer>same ami for all 5
coincidence? i think not!
<eddies1>i usually run one of the amazon linux amis or one of the alestic ones13:16
<cbeer>got an ami id for one of those?13:18
i'd grab a 64-bit. ebs backed or instance depends on your preference, i guess13:19
and us-east is usually cheapest
or at least, last time i checked
<cbeer>ah that's a much better listing than the page i was trying to read
success! thanks13:23
(now i remember why i like puppet.. doing all this sysadmining to get all the basic stuff in place is annoying)13:31
ok, so.. to spin up some more instances (bwahhaha)13:37
* cbeer watches the beer money draining away
i keep forgetting how awesome aws really is when it works right.13:42
barmintor: should i ping ajs6f about setting up spring for the modeshape project, or is it maybe something in my grasp?13:44
if you want to use xml to configure your infinispan storage, here's the XSD. have fun!13:55
<fasseg>heh that article on the data set now looks like my sed fu is strong...
<- imposter
so I'm officialy off for the week end but continue idling here.... if you wanna reach me, google talk might be best though14:01
<cbeer>much better: https://github.com/ModeShape/modeshape/blob/master/modeshape-jcr/src/test/resources/config/thorough-with-desc-repo-config.json14:13
now to just find something similar to infinispan..
<fasseg>eddies: I noted that im still in the schedule for sprint 3 2/6/13 -> 2/19/13, although I'm gonna be enjoy rome for a week from 11th to th 15th..14:35
<barmintor>cbeer: I have yet to identify a task that seems out of your grasp14:37
<cbeer>fasseg: that's also during code4lib, so i bet we'll not be working too hard either14:38
barmintor: true, true.. but is it a task i should take on, or merely suggest to ajs6f, who will solve it instantly?
<fasseg>hmm that's an argument
<barmintor>cbeer: probably the latter
ok, supposedly i have a infinispan cluster of 1 node now14:39
so now for some more important matters still havent finished FTL...14:45
<cbeer>and a cluster of 2 nodes, but it doesn't work.
null pointer errors, how i loath you.
<fasseg>put a "thee" there and it almost sounds like an epigram..14:46
<barmintor>I love null pointer errors. They're the only errors that tell you exactly what's wrong and where it is wrong.14:49
<fasseg>still, the fiddling through the stack part is a chore...14:50
grml steam for linux wont install FTL14:52
* fasseg leaves14:55
<cbeer>barmintor: where, yes. what.. well, there was a null pointer, true.14:56
but the 100 line stack trace doesn't help me know what i've done wrong
* fasseg joins
<barmintor>cbeer: you've called a method on a variable with a null value, of course! :P14:57
<cbeer>"java.lang.IllegalArgumentException: magic number 11851 is not available in magic map"14:58
that's a good one.
<barmintor>that's classic.
you're doing some kind of file characterization?14:59
<fasseg>don't say it out loud, god knows what might happen
<cbeer>apparently that error means i'm using different versions of infinispan.15:02
i don't think much of this is a reflection on their clustering approach though. just my ignorance (and their lack of excellent documentation)15:08
<barmintor>oh, I get it. that sucks.15:25
<cbeer>ok, doing this out on aws isn't getting me anywhere fast. i'm going to shift gears and get a hydra head deployed there15:29
and try again locally i guess
i still don't think it's a problem with modeshape/infinispan/etc. i'm just doing something wrong15:30
they have some example projects i'll play with
* JasonDGI leaves15:34
<barmintor>this whole HintProvider architecture seems a little broken16:01
there's only storage hints, no retrieval hints
and Akubra doesn't know about hint providers, only maps of hint strings16:02
<fasseg>so you cant get hints from the backendwhen fetching data via akubra?16:03
<barmintor>and HintProvider isn't part of the the v3 api for DOManager, so DefaultDOManager just magically gets it from Spring.
fasseg: that's right
<barmintor>fasseg: on retrieval, all the info must be encoded in either the dsid or the dslocation16:04
<cbeer>http://ec2-107-20-81-44.compute-1.amazonaws.com:3000/ <-- hydra-head running on top of the modeshape prototype.
<barmintor>awesome cbeer++
<fasseg>that sounds hacky..
<barmintor>fasseg: yes, it's half baked. Though Akubra itself is v0.4.0, right? :)16:05
so 40% baked.16:06
* barmintor sighs
<fasseg>yes that's true, but it should at least be feature complete ;)16:10
although I guess it is via the dsId and dsLocation
<barmintor>it's pretty bold to expect that any akubra storage module will have a unique dsLocation URI pattern16:11
<cbeer>barmintor: supposedly, when you register new storage modules, you are supposed to give them a unique URI16:12
but still...
<barmintor>cbeer: you give the blobStore a URI, but that doesn't help you pick which blobStore based on the dsLocation URI16:13
unless there's an undocumented meta-requirement that the blobStore URI is the beginning of any dsLocation URIs16:14
but that's also pretty fragile
ok, that's mean. But this is a problem that should be fixed in our project if we want to support real policy-driven storage16:15
<barmintor>but it does mean we have to fork fcrepo to do a glacier demo that can retrieve data16:20
or at least just implement a subclass of AkubraLowLevelStorage16:21
and also DefaultDOManager16:22
but these are new tickets
BRB, pushing to github and closing a ticker
<cbeer>ah, maybe here's what I want for the modeshape prototype infinispan configuration:16:32
The ClusterCacheLoader is a cache loader implementation that retrieves data from other cluster members.
another load testing candidate: http://grinder.sourceforge.net/16:33
maybe we should load test the load testers.
and pick the one that gives the numbers we like the best.16:34
i should also read this:https://docs.jboss.org/author/display/ISPN/Cross+site+replication16:36
ok, taking a break. will bang my head on this more later. maybe with beer-in-hand.16:37
* fasseg leaves19:00
* github-ff joins19:40
[ff-modeshape-prototype] cbeer created clustering (+1 new commit): http://git.io/pcXpyQ
ff-modeshape-prototype/clustering 8bf590c Chris Beer: try some clustering..
* github-ff leaves
* github-ff joins20:38
[ff-modeshape-prototype] cbeer fast-forwarded master from e48993c to 2cb454d: http://git.io/-juFIw
* github-ff leaves
* github-ff joins20:39
[ff-modeshape-prototype] cbeer opened pull request #13: Initialize the repository on servlet load, or clustering may get unhappy... (master...context-listener) http://git.io/WV2Ejw
* github-ff leaves