Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* eddies leaves01:20
* kaarefc joins02:40
* kaarefc leaves03:06
* kaarefc joins03:39
* nbanks joins04:29
* nbanks leaves04:49
* kaarefc leaves05:13
* nbanks joins05:19
* nbanks leaves05:25
* kaarefc joins05:31
* kaarefc leaves05:35
* kaarefc joins05:36
* nbanks joins05:40
* nbanks leaves07:48
* avessey joins
* avessey leaves07:55
* avessey joins08:02
* kaarefc leaves08:27
* VincentNG joins08:36
* nbanks joins08:53
* jcoyne joins08:55
* kaarefc joins08:56
* kaarefc leaves09:28
* jcoyne leaves10:06
<pivotal-bot>Jonathan Green edited "Add list of minimum required RDF predicates required to make Islandora happy to a ticket" https://www.pivotaltracker.com/story/show/4586486710:16
Chris Beer edited "Add list of minimum required RDF predicates required to make Islandora happy to a ticket" https://www.pivotaltracker.com/story/show/4586486710:21
Chris Beer edited "Create a rdf generator that serializes properties into the minimum required set of properties that make Islandora happy" https://www.pivotaltracker.com/story/show/45864541
* barmintor joins10:23
<cbeer>hm. so fcrepo-audit is all asychronous and just listens to the eventbus, huh?10:29
(also, this looks interesting: https://github.com/bennidi/mbassador)10:32
i suspect for things like self-healing, we want sychronous audit logging
* jcoyne joins10:40
barmintor: do you know a guy named Schyler?
<barmintor>Duveen?
<jcoyne>Used to work at Columbia.
<barmintor>yeah, I know him
he seemed pretty sharp. Worked in another department, though10:41
* fasseg joins10:47
* eddies joins10:57
* eddies leaves
* eddies joins
barmintor: coming?11:01
<barmintor>yep yep
<VincentNG>can u hear me?11:03
internet is bad here today, can you invite me? 404-498-0384
<cbeer>will do
<VincentNG>i'm at the office
<cbeer>barmintor: https://github.com/futures/fcrepo4/commit/8bbd68a6ba8ebd53d16c54078de57c5dd72aa68b11:06
<barmintor>cbeer: ok, do we want the correction to happen on the fcrepo side or the montior/service side?11:07
<cbeer>i have no idea.
<barmintor>I *think* the former
YES11:08
eddies: yes11:09
<cbeer>YES NO FILENOTFOUND11:10
<pivotal-bot>Benjamin Armintor added "FCRepo fixity check does blocking heal" https://www.pivotaltracker.com/story/show/4638298911:20
Benjamin Armintor edited "FCRepo fixity check does blocking heal" https://www.pivotaltracker.com/story/show/46382989
Benjamin Armintor added "FCRepo does asynch heals, and Reporting service logs that" https://www.pivotaltracker.com/story/show/4638306111:21
Benjamin Armintor edited "FCRepo does asynch heals, and Reporting service logs that" https://www.pivotaltracker.com/story/show/46383061
Benjamin Armintor edited "FCRepo fixity check does blocking heal" https://www.pivotaltracker.com/story/show/4638298911:22
Benjamin Armintor edited "Call self-healing Datastream#runFixityAndFixProblems when fixity problems are discovered" https://www.pivotaltracker.com/story/show/4631616111:23
Edwin Shin edited "Call self-healing Datastream#runFixityAndFixProblems when fixity problems are discovered" https://www.pivotaltracker.com/story/show/46316161
<eddies>jonathangee: got a min to talk?11:29
<cbeer>fasseg: ok, here's my first question, and is probably a lack of understanding about jax-rs
oh, or maybe what's on the server isn't the latest code11:30
<jonathangee>sure eddie. do you mind waiting 5 min? i've got someone else here right now
<cbeer>nope, i guess it is. ok fasseg, here's the question:
<jonathangee>i can ping you on Skype if that works
<cbeer>i see this url in the web ui:11:31
http://fcrepo4.fedora4lib.org/fcrepo-fixity/static/details.html?id=4
i don't see how that gets routed in jax-rs
<fasseg>oh yeah evil JS hack....it in the js file..lemme check
o a JS takes the URL param and creates a new ajax request to a different endpoint
<cbeer>ah ha11:32
<fasseg>Results.java:L44:
@Path("/details/{recordId}").....
<barmintor>eddies: did I tell you about this NEFUG groups that wants me to talk to them about FF later in the Spring?11:33
<fasseg>so you can do a http://localhost:8080/fixity/details/<;id>
<eddies>barmintor: i saw you mention it on irc the other day…in boston, i think you said?
<barmintor>Springfield11:34
<cbeer>fasseg: e.g. http://fcrepo4.fedora4lib.org/fcrepo-fixity/results/details/4 ?
<eddies>i didn't even know there *was* a NE FUG
<fasseg>right..
<cbeer>fasseg: and that's a 404 for me
<fasseg>it an inlines JS in detailas.html
lemme see
<cbeer>so now i'm back to being confused.
thanks
<fasseg>can you try: http://fcrepo4.fedora4lib.org/fcrepo-fixity/rest/results/details/411:35
(+/rest/)
<cbeer>cool
fasseg: how is the /rest/ getting in there?
<fasseg>web.xml11:36
since "/static/" should not go through the resteasy servlet
<cbeer>oh, ok.
so, ideally (but done this was for expedience, etc, etc):
the content at http://fcrepo4.fedora4lib.org/fcrepo-fixity/static/details.html?id=4 would just be a text/html response from http://fcrepo4.fedora4lib.org/fcrepo-fixity/rest/results/details/4, right?11:37
<fasseg>but then it's no longer static content and you will invoke the java servlet for simple static content...11:38
which I think is kinda unneccessary...
<cbeer>aren't you anyway? when you go and request the data?
<fasseg>not static content, just some json dynamic generated but the page and the script itself are served thorugh the HTTP connector11:39
but it would be simple to just create an html representation at this point11:40
<cbeer>ok, i think that's what i was expecting to see. but it sounds like i could be wrong to expect that, if there are serious performance considerations11:41
<fasseg>well serious.....
<cbeer>in any case, i can live with it for now
<fasseg>premature optimisation is the...
but for me and the prjojects i've done it's not unusual to have a pure rest/xml rest enpoint and one for html representations.11:42
but of cause it would be more intuitive to server the html contetn at the same enpoint...11:43
hmm lemme just fiddle a second...
<cbeer>eddies: we need that service registry soon if we're going to keep building these very external services11:49
i'm going to write some hacky code for tomorrow, but it's a mess.
e.g. where is the service, what authn do i need, etc
<pivotal-bot>Jonathan Green added "multi-tenancy in single fedora" https://www.pivotaltracker.com/story/show/4638639511:53
Jonathan Green edited "multi-tenancy in single fedora" https://www.pivotaltracker.com/story/show/46386395
* kaarefc joins
<fasseg>hmm it seems in order to have resteasy serve static content a plugin like "https://code.google.com/p/htmleasy/" is neccessary..11:58
this sucks.
thought View and the like was part of JAX-RS but obviously not.
cbeer, eddies, ben: Include "htmleasy" to have the RESTEasy controller serve html pages? Although this seems not as well supported as jboss' software, it looks like an active community project...https://code.google.com/p/htmleasy/ ?12:05
it would be more intuitive to request the same endpoints with an html accept header, as pointed out by chris...12:06
but the static html would be routed through a JSP processor..
i.e. the result view is exposed at /static/results.html while the corresponding rest endpoint is /rest/results/ ....12:10
and we would have to add a meven repo at: https://code.google.com/p/htmleasy/wiki/UsingMaven12:13
*blargh*
hmm there's also a way to use spring-mvc...but why then use RESTEas in the first place?12:15
hmm what's resteasy-html12:16
<cbeer>i thought we were using velocity somewhere else?12:17
<fasseg>hmm not seen that...12:18
but resteasy-html seems to have a View class...
and it's a jboss thing
<barmintor>fcrepo-http-api is pulling in velocity as a dep12:19
<VincentNG>velocity is being used at /rest/search12:26
<cbeer>jax-b sure does make some horrific xml12:30
* avessey leaves12:43
* ff-github joins12:48
[fcrepo4-demo-hydra-app] cbeer pushed 1 new commit to master: http://git.io/rSTNYQ
fcrepo4-demo-hydra-app/master 6da3182 Chris Beer: fixity status report.
* ff-github leaves
<cbeer>barmintor: i'm not going to make a ticket until i know what you've done, but the current fixity report from fcrepo-fixity doesn't include the information to satisfy my ticket12:50
e.g. the external storage uri we're providing
<barmintor>cbeer: ok
I've got the one error fixed, let me push it up
<cbeer>and if you make the jaxb output more parsible i'll be pleasantly surprised12:51
right now we have fixity-result/successes/successes and fixity-result/errors/errors, which feels pretty funny to write xpaths for
(and i see a status attribute crying out to be useful)12:52
but maybe your work will just implicitly fix that
<barmintor>yeah, I'm working on a status thing now
though the only impl that makes sense to me will be kind of crappy in xml12:53
ie, some kind of bit switch where 0 = success, 1 = repaired, 2 = bad checksum, 4 = bad size
but suggestions are welcome12:54
<bljenkins>Project fcrepo-fixity build #42: SUCCESS in 1 min 36 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/42/12:55
<fasseg>cbeer: I could rename the XMLAttributes to something like fixity-result/error-list/errors or sth.
<bljenkins>Project fcrepo-fixity build #43: SUCCESS in 1 min 22 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/43/12:56
<barmintor>the responsibility for fixity and repair in kernel has gotten a little screwy
<fasseg>but you can also use a Jaxbunmarshaller for those xml/json files and it will create a Fixity result Java bean for you, if you dont absultely need xpath expressions
<barmintor>LLStoreService does fixity, Datastream does repair, etc.12:57
<cbeer>barmintor: i think the fixity calculation can be moved up to Datastream
that was just a convenient place to stash it
or, a new class.
<barmintor>cbeer: yeah, that seems right12:58
* kaarefc leaves13:00
* kaarefc joins
<cbeer>barmintor: from futures6:13:01
Caused by: java.lang.ClassNotFoundException: org.fcrepo.services.fixity.model.FixityResult
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1701) ~[tomcat-catalina-7.0.26.jar:7.0.26]
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1546) ~[tomcat-catalina-7.0.26.jar:7.0.26]13:02
at org.springframework.util.ClassUtils.forName(ClassUtils.java:258) ~[spring-core-3.2.0.RELEASE.jar:3.2.0.RELEASE]
at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:291) ~[spring-core-3.2.0.RELEASE.jar:3.2.0.RELEASE]
... 66 common frames omitted
your doing?
<barmintor>it must be, but not expected. I think it needs new snapshots
where the heck is there a spring config with the old classname?13:10
oh hell, ok13:12
<bljenkins>Project fcrepo-fixity build #44: SUCCESS in 1 min 1 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/44/13:15
<barmintor>sorry about that13:18
<pivotal-bot>Chris Beer added comment: "http://hydra.fcrepo.org/files/td96k2517/fixity" https://www.pivotaltracker.com/story/show/4569138513:26
Chris Beer edited "Expose object fixity state in Hydra" https://www.pivotaltracker.com/story/show/45691385
Chris Beer delivered "Expose object fixity state in Hydra" https://www.pivotaltracker.com/story/show/4569138513:27
Chris Beer added comment: "Exposes as much information as available from the fixity service, at least." https://www.pivotaltracker.com/story/show/45691385
<eddies>hey guys. i've been on conference calls for the past 2.5hrs :P but back now14:27
anything i can help/answer with?
fasseg: i think ajs6f, vincentng at least have all observed that you need some kind of view technology to pair w/ jax-rs if you want html14:28
and vincentng did a strawman with resteasy+htmleasy
cbeer: is the kitchen-sink that's deployed to futures6 configured to use the storage-alt (i.e. the redundant store)?14:29
<cbeer>eddies: no, i don't think it is.
but it could be.14:30
<eddies>well, it should be, else we can't demonstrate the self-healing feature
i'll add a chore
<cbeer>thanks
<barmintor>cbeer: I think the way to simplify a lot of the stack is to push the generation of any given status all the way down to the LowLevelCacheEntry14:33
<cbeer>barmintor: for fixity?14:34
<barmintor>yeah
<pivotal-bot>Edwin Shin added "Configure kitchen-sink on futures6 to use redundant store" https://www.pivotaltracker.com/story/show/46402651
Edwin Shin edited "Configure kitchen-sink on futures6 to use redundant store" https://www.pivotaltracker.com/story/show/46402651
<cbeer>barmintor: i'm nervous about pushing logic to the lowlevelcacheentry14:35
<barmintor>that is really just pushing the checksum gen down to the class that knows about the bytes
I'll push this stuff up in a branch so you can see what I mean14:36
<cbeer>barmintor: ok. in a meeting now, but i'll look after
<barmintor>and then I'll cheange it if you think it's a bad diea
<pivotal-bot>Edwin Shin edited "FCRepo does asynch heals, and Reporting service logs that" https://www.pivotaltracker.com/story/show/46383061
Edwin Shin added comment: "I'm getting an authorization error (You are not authorized to access this page) when I visit that url (logged..." https://www.pivotaltracker.com/story/show/4569138514:41
* github-ff joins
[fcrepo4] barmintor created lowlevel-fixity (+1 new commit): http://git.io/KVtoZw
fcrepo4/lowlevel-fixity 316a8c5 Benjamin Armintor: refactoring checksum verification to support self-healing and reporting on better
* github-ff leaves
<pivotal-bot>Edwin Shin edited "Expose object fixity state in Hydra" https://www.pivotaltracker.com/story/show/45691385
<bljenkins>Project fcrepo-kitchen-sink build #79: SUCCESS in 2 min 24 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-kitchen-sink/79/14:49
Project fcrepo-fixity build #45: SUCCESS in 1 min 51 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/45/
* nbanks leaves14:57
<fasseg>eddies: yeah but there is a resteasy-html maven dependency which im testing atm...15:13
<eddies>fasseg: got a link for that?
<fasseg>but maybe I'll have to drop the /static/results.html path and use only /rest/results, Accept="text/html"15:14
<eddies>nm, this one, right?: http://search.maven.org/#artifactdetails|org.jboss.resteasy|resteasy-html|3.0-beta-3|jar
<fasseg>nah just found the maven dependency with about 4 classes in them, one called View which lets you do a new View("path/to/resource.html);
yeah but 2.5.x15:15
but since I would not want to change to JSPs i might have to create a resource which returns the html files traigt from dsk using a StreamOutput15:17
and the js/css files of course...
and if a resource like a css or a js file should be available at different paths (e.g. /rest/results and /res/results/details) this leads to issues when resolving the path of the files, since you can't do it relatively15:19
and if you're in a different context than root on your webapplication you can't use absolute ones..15:20
<cbeer>eddies: re: https://www.pivotaltracker.com/projects/684825#!/stories/4569138515:28
you need to be an editor of an object to see the fixity
so.. make your own object, i guess
<eddies>well, i think we should relax those permissions in the hydra head for now15:30
but in any case, it's not an individual object's fixity report that the ticket is for anyway
<cbeer>oh, i misread the ticket then15:31
<eddies>it's a summary report of repository fixity check
<cbeer>so.. fasseg's interface, done in ruby?
<eddies>fasseg's report data consumed/digested and reported in a hydra head
same same but different, i suppose15:32
as they say here in singapore
<cbeer>i don't want to try to parse all that jax-b nasty xml in ruby15:33
* ajs6f joins15:37
cbeer: MBassador looks quite cool.
cbeer/barmintor: Is there some kind of beef with the JAXB stuff for the fixity messages? I'm willing to bet that JAXB can do what you want, but it might not be obvious how. Want another pair of eyes?15:38
<barmintor>ajs6f: I haven't actually looked at the XML
and I'm trying to figure out this other thing right now
<ajs6f>Yeah, it was cbeer I heard complaining. :)
<cbeer>ajs6f: yeah, the fixity xml is.. unpleasant.. to parse in ruby... and eddies apparently wants fasseg's interface exposed through the hydra head15:42
so.. i guess xml parsing it is
<eddies>theres supposed to be json
<ajs6f>cbeer: what, exactly, is unpleasant about it? Maybe we can improve it? Or, if it's a JAXB output, can't you just use the JSON instead?
<eddies>what he said ^^15:43
afk a min
<cbeer>i'm not anticipating the json being any better
it'll be the same structure, different format.
<ajs6f>So let's improve the structure! What do you find objectionable about it now?15:44
<cbeer>being out of permgen space, for one.15:45
<ajs6f>Erm— is that definitely related to XML production?15:46
<cbeer>nope, but definitely stops me from telling you what i don't like about the xml15:47
http://fcrepo4.fcrepo.org/fcrepo-fixity/rest/results
<ajs6f>I just loaded that URL fine...?
<cbeer>i'm not sure why successes and errors need to be in different wrappers (and how that'll mesh with barmintor's work)15:48
(yeah, because i restarted tomcat)
successes/successes is a little funny
<barmintor>Yeah, there's some weirdness in the labels that I'm not correcting
let me get this other thing working first
<ajs6f>So you want something with fewer containers… and successes and failures in different elements, but in the same container?
<cbeer>well, seems like there's a lot of redundant data there, at least.. but, I forgot, barmintor is in the middle of some refactoring15:49
so i should just wait and see what he's doing
<ajs6f>Okay. Just one tip I'll throw in ahead of time: remember that in JAXB, you can annotate accessor methods as well as or instead of fields. So if you need to get something more complex going, it's often easiest to use some simple sorting or order logic right there in the JAXB-annotated class. That way you keep your data modeled how you like it and just spit it out differently.15:51
<barmintor>instead of. If you annotate the field and the accessor you'll get errors15:52
cbeer: https://github.com/futures/fcrepo4/commit/316a8c547be185924aee98fd90ae7b0bfd9b6ead
that's the lowlevel cache entry thing I was talking about
<ajs6f>The same field, sure. I meant you can use field-annotation for some things and accessor-annotation for others.15:53
* nbanks joins15:54
<cbeer>barmintor: yeah. i'm not thrilled, but i'm also going to go to lunch and if it got merged into master while i was gone, i wouldn't complain or try to revert it.
<barmintor>:P15:55
<cbeer>(mainly, i don't like putting functionality into a class that is really a massive hack)
but, it's there. and now that it knows about objects, i guess we should just use it.
* nbanks leaves16:00
* VincentNG leaves16:16
* ajs6f leaves16:38
* nbanks joins16:52
* fasseg leaves16:54
<cbeer>barmintor: have you looked much at the DefaultDatabaseService class in fcrepo-fixity?16:57
* nbanks leaves
<barmintor>I'm looking at it now
I'm trying to teach it about errors vs repairs
<cbeer>barmintor: ok. so, one of the things eddies wants is " i) # of resources verified"16:58
<barmintor>ok
<cbeer>the fixity app seems to keep a counter of successes and failures of checks
<barmintor>is there a ticket for it?
yes, it does
<cbeer>(it's all rolled into the hydra ticket: https://www.pivotaltracker.com/story/show/45691385)
<pivotal-bot>feature: Expose object fixity state in Hydra (rejected) / owner: Chris Beer
* kaarefc leaves16:59
<cbeer>if this was just sql, i'd say successes and failures could just be calculated from the result data that it looks like we're also storing, righT?
is the counter just an optimization? or life with hibernate?17:00
<barmintor>yeah, that's what it does
<cbeer>barmintor: i see that in getSuccessCount/getErrorCount17:01
but there's the addStat thing for the daily counts
<barmintor>oh, yeah
so you're wondering why not just go to the db for that, too17:02
<cbeer>yeah.. because then i'd just do the same thing, but de-depe by PID and get eddies' stat.
s/depe/dupe/
(and trust the backend store to be indexed appropriately, etc)
<barmintor>I don't know of a reason off the top of my head not to do that
<cbeer>ok, i just wanted to make sure there wasn't a hibernate reason not to do that17:03
i still don't understand why it's important to poke this through hydra if we're conceiving this fixity app as its own stand-alone, repository agnostic thing.17:05
* nbanks joins17:07
<bljenkins>Project fcrepo-fixity build #46: SUCCESS in 59 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/46/17:08
* nbanks leaves17:12
<bljenkins>Project fcrepo-fixity build #47: SUCCESS in 1 min 2 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/47/17:13
* ff-github joins17:14
[fcrepo-fixity] none pushed 3 new commits to master: http://git.io/HkaqHg
fcrepo-fixity/master 242e4cb Benjamin Armintor: hibernate dserialization must be supported
fcrepo-fixity/master a262d83 Benjamin Armintor: non-test spring configs should be updated, too
fcrepo-fixity/master 33e3369 fasseg: added property reference for 'org.fcrepo.fixity.brokerurl' to README
* ff-github leaves
* ff-github joins
[fcrepo-fixity] fasseg pushed 1 new commit to master: http://git.io/j1bpLw
fcrepo-fixity/master b673fb1 fasseg: added some markdown to README
* ff-github leaves
<bljenkins>Project fcrepo-fixity build #48: SUCCESS in 1 min 20 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/48/17:16
* ff-github joins
[fcrepo-fixity] fasseg pushed 1 new commit to master: http://git.io/Vov-LQ
fcrepo-fixity/master 8cde5ec fasseg: markdown backticks were the wrong way around in README:/
* ff-github leaves
<bljenkins>Project fcrepo-fixity build #49: SUCCESS in 1 min 7 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/49/17:17
* jcoyne leaves17:20
* fasseg joins17:24
<pivotal-bot>Chris Beer added "Aggregate fixity results by resource" https://www.pivotaltracker.com/story/show/4641744717:26
Chris Beer added "REST API to filter fixity results" https://www.pivotaltracker.com/story/show/46417497
Chris Beer edited "REST API to filter fixity results" https://www.pivotaltracker.com/story/show/46417497
Chris Beer edited "Aggregate fixity results by resource" https://www.pivotaltracker.com/story/show/46417447
<cbeer>i give up on trying to do those myself. hibernate is beyond me right now.17:27
i also don't know what the correct behavior in those reports are..
say an object fails a fixity check, and then an hour later passes..
do we include the failure in the failure report?17:28
<fasseg>hm iguess eddies answer would be the often heard "If i was an admin, I would like to know when a self heal occured"17:30
can I help you out with hibernate
<cbeer>see those two tickets17:31
i think the filtering one is "easy"17:32
i suspect i'm just having trouble getting all the data through the layers in the right formats
<fasseg>so shall i create an endpoint for "- count of unique resources verified"17:33
<cbeer>yes please. and i think it should include counts for failures
and the hydra ticket suggests that i should be able to filter it by date
https://www.pivotaltracker.com/story/show/45691385
<pivotal-bot>feature: Expose object fixity state in Hydra (rejected) / owner: Chris Beer
<cbeer>that ticket
<fasseg>hmm..there's already "/rest/statistics/general" i could add those two properties...17:35
it'll just give you the error and success count atm but I'll just add the uniw resources
*uniqe
<cbeer>that'd be good enough.17:36
<fasseg>ok ....on it
<cbeer>and if we could narrow that by time, even better.
<fasseg>so add start/end params or rather start/length...17:37
* github-ff joins
[fcrepo4] barmintor merged lowlevel-fixity into master: http://git.io/lQvKUA
* github-ff leaves
<barmintor>cbeer: it'll be easier to refactor with the functionality in one place, if it's any comfort :P17:38
fasseg: do you have any hints for testing the fixity webapp?17:39
<fasseg>arg much dreaded question...
there seems to be a whole paragraph on mocking in the resteasy docs but i haven't read it so far....shame on me17:40
http://docs.jboss.org/resteasy/docs/2.3.5.Final/userguide/html_single/index.html#RESTEasy_Server-side_Mock_Framework
<barmintor>ok
<fasseg>but they're just mocking the resources, so these are not part of the tests, basically the same i did in fixity-core when mocking the fedora-client..17:42
<bljenkins>Project fcrepo-fixity build #50: FAILURE in 23 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/50/
Project fcrepo-kitchen-sink build #80: SUCCESS in 1 min 53 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-kitchen-sink/80/17:43
* ff-github joins17:48
[fcrepo-fixity] barmintor pushed 1 new commit to master: http://git.io/t-7n_w
fcrepo-fixity/master 4ac087a Benjamin Armintor: pulling in repair stats
* ff-github leaves
<barmintor>the fixity webapp just has to be broken for a bit17:49
<bljenkins>Yippie, build fixed!
Project fcrepo-fixity build #51: FIXED in 1 min 9 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/51/
<barmintor>is eddies planning on demoing the self-healing thing?17:50
<cbeer>i'm not sure how he plans to demo it
<barmintor>me either
* jcoyne joins17:52
<barmintor>fasseg: I may need to get some advice from you about how to blow away the db that fcrepo-fixity is running on and rebuild it17:53
or blow it away, and just run the report I guess
<fasseg>sure.....it's a hsql file db at the moment...
just deleting /tmp/fixity-results.db should work...17:54
* jcoyne leaves
<fasseg>the url in the context-fixity-core.xml is: jdbc:hsqldb:file:/tmp/fixity-results.db;shutdown=true
there also a example config for postgresql in there, so you can that..17:55
*use that
* travis-ci joins17:59
[travis-ci] futures/fcrepo4#255 (master - 316a8c5 : Benjamin Armintor): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/4971b28ce9de...316a8c547be1
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/5607894
* travis-ci leaves
<pivotal-bot>Chris Beer started "Configure kitchen-sink on futures6 to use redundant store" https://www.pivotaltracker.com/story/show/4640265118:07
Chris Beer finished "Configure kitchen-sink on futures6 to use redundant store" https://www.pivotaltracker.com/story/show/46402651
<fasseg>barmintor: when trying to queue objects in the fixity service i get this one now: https://gist.github.com/fasseg/5191251
have you seen this before?
<barmintor>Like I said, the webapp is going to be broken until it's fixed18:08
<fasseg>oh sry missed that
<barmintor>yeah, I'll get it running again and then start working on some tests
(I realize that's the wrong order, but I don't know what else to do)18:09
<fasseg>no worries, i can use a break ;=)
but please ping me when it's running again..
so i can add the stats for cbeer18:10
* ff-github joins18:16
[fcrepo-fixity] barmintor pushed 1 new commit to master: http://git.io/kXDF6g
fcrepo-fixity/master fb67761 Benjamin Armintor: working repairs into the json
* ff-github leaves
<bljenkins>Project fcrepo-fixity build #52: SUCCESS in 1 min 5 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/52/18:17
* ff-github joins18:22
[fcrepo-fixity] barmintor pushed 1 new commit to master: http://git.io/8d7JXg
fcrepo-fixity/master d1ecb0f Benjamin Armintor: js syntax error
* ff-github leaves
<fasseg>missed the commas after the array elements in fcrepo-fxity error
ah you saw it...
<bljenkins>Project fcrepo-fixity build #53: SUCCESS in 1 min 16 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/53/18:23
<barmintor>well, damn18:24
<pivotal-bot>Frank Asseg added comment: "grabbing the task to implement extended general statistics" https://www.pivotaltracker.com/story/show/4641744718:25
Frank Asseg edited "Aggregate fixity results by resource" https://www.pivotaltracker.com/story/show/46417447
<fasseg>hmm and id would probably make sens to pull the db config from core to the webapp so ppl can edit the db configuration....18:27
<barmintor>I'm perplexed. The fixity-core stuff does have tests, and they were fine18:29
ah, couldn't connect to the db18:30
Database lock acquisition failure: lockFile: org.hsqldb.persist.LockFile@ffb0c578[file =/tmp/fixity-results.db.lck, exists=true, locked=false, valid=false, ] method: checkHeartbeat read: 2013-03-18 22:29:50 heartbeat - read: -7896 ms.)
<fasseg>different settings in test and main context?18:31
<barmintor>^^ look familiar?
<fasseg>hmm do you have a shutdown=true after the url for hsqldb?
this happens when the file db is not properly closed i think
or somthing hung from the test and holding a lock on the file?18:32
<barmintor><property name="url" value="jdbc:hsqldb:file:/tmp/fixity-results.db;shutdown=true" />
<fasseg>looks fine
and when you remove the lock and rerun the same issue happens?18:33
<barmintor>I'm looking up my password to futures6
<fasseg>oh I thought you were running a local instance..
i can check there...18:34
ssh is so very annoyingly slow to the futures boxes from here...18:35
scheduled a new build to try18:37
<barmintor>ok
<fasseg>but it seems the db got a hit...-> valid=false...18:38
<bljenkins>Project fcrepo-fixity build #54: SUCCESS in 57 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/54/
<fasseg>now the db error seems to be gone, but this one happens: https://gist.github.com/fasseg/519150718:40
but still this one in there: SQL Error: -5501, SQLState: 4250118:41
<barmintor>can you post the rest of that stacktrace?18:42
<fasseg>sure...
oh you mean the sql thingie?
that's a warning message18:43
<barmintor>No, the rest of the unabel to deserializestacktrace
<fasseg>ok..
wait a sec: ERROR 22:38:36.268 (SqlExceptionHelper) user lacks privilege or object not found: THIS_.REPAIRCO4_3_0_18:44
^^ that one...
I thought hibernate was set to update schema
i guess the db did get inconsistent....18:45
ill just delete it completely....
new stacktrace: https://gist.github.com/fasseg/519159618:49
but the db errors are gone it seems
<barmintor>well, there's a gap in testing18:53
guh, how the hell do you test what classes a jaxb unmarshaller knows about?19:05
<fasseg>hmm normally the JaxbContext knows about the classes it can marshall/unmarshlal19:07
but nothing on the context to retreive the known classes19:08
marshaller seems to have a property "classesToBeBound" so you can maybe acces it via reflection?19:09
hmma mybe this one: http://docs.oracle.com/javaee/5/api/javax/xml/bind/JAXBIntrospector.html19:10
you can get the introspector from the context and check if an object is bound via isElement() i think....19:11
but never done this before
but i guess the isElement() method might return true for any class which has a @XmlRootElement, whether it's bound to the context or not...19:14
<bljenkins>Project fcrepo-fixity build #55: SUCCESS in 1 min 4 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/55/19:15
Project fcrepo-fixity build #56: SUCCESS in 1 min 3 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/56/19:23
<barmintor>God, I hate you hibernate19:25
I have to go home, I'l fix this tonight
<fasseg>ok see you ben!19:26
* fasseg leaves
* barmintor leaves19:33
* jcoyne joins20:07
* jcoyne leaves20:17
* pivotal-bot leaves20:50
* cbeer leaves
* cbeer joins20:58
* pivotal-bot joins21:08
<cbeer>hello21:09
<pivotal-bot>Hello, cbeer
* pivotal-bot leaves21:13
* pivotal-bot joins
<cbeer>hello
<pivotal-bot>Hello, cbeer
<cbeer>cinch++
karma21:14
pivotal-bot++
hm
* pivotal-bot leaves
* pivotal-bot joins21:18
<cbeer>hello
<pivotal-bot>Hello, cbeer
<cbeer>pivotal-bot++
!remind us to test this bot in 1 minute21:19
<pivotal-bot>cbeer: Sorry, I don't quite understand the phrasing
<cbeer>!remind us to test this bot in 1 min
<pivotal-bot>cbeer: I will remind this channel about "test this bot" at 2013-03-18 18:20:24 -0700
test this bot21:20
* pivotal-bot leaves21:30
* pivotal-bot joins21:32
<cbeer>@karma
hello
<pivotal-bot>Hello, cbeer
<cbeer>pivotal-bot++21:33
!@karma
* pivotal-bot leaves21:35
* pivotal-bot joins21:36
<cbeer>pivotal-bot++21:37
@karma
<pivotal-bot>Highest karma: pivotal-bot (0)
Lowest karma: pivotal-bot (0)
<cbeer>pivotal-bot++
pivotal-bot++
@karma
<pivotal-bot>Highest karma: pivotal-bot (0)
Lowest karma: pivotal-bot (0)
* pivotal-bot leaves21:40
* pivotal-bot joins21:42
<cbeer>pivotal-bot++
@karma
* pivotal-bot leaves21:43
* pivotal-bot joins
<cbeer>pivotal-bot++
@karma
<pivotal-bot>Highest karma:
Lowest karma:
* pivotal-bot leaves21:44
* pivotal-bot joins21:45
<cbeer>pivotal-bot++
@karma
<pivotal-bot>Highest karma: pivotal-bot_inc (1), pivotal-bot (1)
Lowest karma: pivotal-bot (1), pivotal-bot_inc (1)
* pivotal-bot leaves21:46
* pivotal-bot joins21:47
<cbeer>@karma
* pivotal-bot leaves21:48
* pivotal-bot joins
<cbeer>@karma
<pivotal-bot>Highest karma: pivotal-bot (1)
Lowest karma: pivotal-bot (1)
<cbeer>eddies++21:49
eddies++
@karma
<pivotal-bot>Highest karma: pivotal-bot (1), eddies (2)
Lowest karma: eddies (2), pivotal-bot (1)
* pivotal-bot leaves
* barmintor joins21:53
* barmintor leaves21:54
* barmintor joins21:59
* pivotal-bot joins22:05
<cbeer>hello22:06
<pivotal-bot>Hello, cbeer
<cbeer>@karma
<pivotal-bot>Highest karma:
Lowest karma:
<cbeer>pivotal-bot++
@karma
<pivotal-bot>Highest karma: pivotal-bot (1)
Lowest karma: pivotal-bot (1)
<cbeer>pivotal-bot--
@karma
<pivotal-bot>Highest karma: pivotal-bot (0)
Lowest karma: pivotal-bot (0)
<barmintor>pivotal-bot is cheating, i think22:07
I am completely unable to ssh to futures622:22
it is futures6.bodleian.ox.ac.uk, itsn't it?22:23
<cbeer>with a special port22:27
pmed.22:28
PATCHES WELCOME22:30
<barmintor>this hibernate + jaxb + jaxrs thing is a nightmare to debug22:40
<cbeer>partly why i gave up earlier.22:44
appears deceptively straightforward, but...
<barmintor>it's also not clear to me how to test this stuff.22:51
I guess hibernate might have it's own test helpers...
"AbstractTransactionalDataSourceSpringContextTests"22:52
<cbeer>FactoryBeanFactory22:57
<barmintor>exactly
* ff-github joins
[fcrepo-fixity] barmintor pushed 1 new commit to master: http://git.io/mDHudQ
fcrepo-fixity/master 0bf2427 Benjamin Armintor: init problems for successes and errors
* ff-github leaves
<bljenkins>Project fcrepo-fixity build #57: SUCCESS in 1 min 30 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/57/22:58
<barmintor>hey, it works, but it says everything is broken!23:01
this is really troubling!23:03
http://fcrepo4.fcrepo.org/fcrepo-fixity/rest/results
<cbeer>hmm..23:05
http://fcrepo4.fcrepo.org/fcrepo/rest/objects/sufia:td96k2496/datastreams/content/fixity
doesn't sounds much better
content looks good though23:06
<barmintor>the computed sum and size is the same for both ds's in that link
kind of suspicious
<cbeer>"hey steering group. we intentionally broke our repository, so you could see what it'd look like"23:07
<barmintor>service with a smile!23:08
<cbeer>the checksum info in http://fcrepo4.fcrepo.org/fcrepo/rest/objects/sufia:td96k2496/datastreams/content/ is calculated from modeshape right? it isn't from the fedora fixity properties?
<barmintor>yeah, that's the stored checksum
it looks like it's just pulling the wrong data for the calculated values23:09
and it's the same wrong data
<cbeer>that's something, at least23:10
<barmintor>which probably means my refactor this afternoon is wrong about where it's pulling the data23:11
<cbeer>oh, yeah. pulling the latest code would probably help me look at it.23:12
hm. the SelfHealingTest still passes23:19
(if you can figure out how to run a test with a different modeshape config, that'd be awfully nice)23:20
<barmintor>the characterization ds is the one that actually has that checksum
http://fcrepo4.fcrepo.org/fcrepo/rest/objects/sufia:td96k2496/datastreams/characterization/fixity23:21
so… some kind of concurrency/object reuse thing
<cbeer>ooh
digest = MessageDigest.getInstance(getContentDigestType());
I don't think MessageDigest is thread-safe?
oh, but you're passing it into my method i guess23:22
<barmintor>I got the idea to pass a clone from somewhere else in the codebase, but that might not work either
<cbeer>yeah, it gets cloned eventually
there goes the only thing i learned23:23
<barmintor>:(23:24
these lowlevel cache entries shouldn't be cached, should they? don't they wrap a single key?23:27
<cbeer>yes, although originally they wrapped just the store.23:28
so maybe something weird went wrong
i'm trying to add another ds to the self-healing test
<barmintor>yeah, I think that's our culprit
we meant to cache the cahcestores, not the entries
see also the cached static members name: cachestores23:29
<cbeer>yup, i broke the test.23:30
<barmintor>good23:31
<cbeer>i do enjoy the java debugger and the ability to reliably jump to a definition/invocation.23:34
<barmintor>ok, the easiest way to test it should be to just yank out the static caching in LowLevelStorageService23:37
and then figure out what/how we can cache the stores afterwards
<cbeer>i think i've done that, and i think i'm still seeing the problem
<barmintor>oh, really?
<cbeer>likely there's something i've missed higher in the call stack23:38
<barmintor>happy to look if you want to push to a branch
<cbeer>oh, no. i'm an idiot.23:40
copypasta.23:41
<barmintor>the bane of us all
<cbeer>there.23:43
* github-ff joins
[fcrepo4] cbeer pushed 2 new commits to master: http://git.io/ho2-8Q
fcrepo4/master 21b259d Chris Beer: update the test to create an extra ds tto replicate a caching bug
fcrepo4/master 5738d3e Chris Beer: fix aforementioned caching problem.
* github-ff leaves
<cbeer>let's see if that makes everything better
@isitdeployedyet?23:46
<barmintor>I'm scared to click, b/c I don't want to tickle that database23:47
maybe we should have a static page at /isitdeployedyet
<bljenkins>Project fcrepo-kitchen-sink build #81: SUCCESS in 3 min 2 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-kitchen-sink/81/23:53
<cbeer>better.
<bljenkins>Project fcrepo-fixity build #58: SUCCESS in 2 min 30 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/58/
<barmintor>better23:54
<cbeer>just had to create some errors for eddies to demo tomororw.
<barmintor>I still need to figure out how to test all this stuff. It's illusory progress23:57
cbeer++ thanks for the help there23:59
* travis-ci joins00:02
[travis-ci] futures/fcrepo4#256 (master - 5738d3e : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/316a8c547be1...5738d3ef60aa
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/5615600
* travis-ci leaves
<barmintor>cbeer: these two tickets are basically the same, so I'm making you the requester on the one I've marked finished (instead of eddies) and deleting the dupe00:05
<cbeer>+1
<barmintor>I think all we have left is refinements to the fixity webapp00:08
so I'm going to go to sleep00:09

Generated by Sualtam