Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* ksclarke leaves01:23
* kaarefc joins02:08
* nbanks joins06:04
* nbanks leaves
* nbanks joins07:13
* nbanks leaves08:52
* gregjansen joins08:55
* kaarefc leaves08:56
* ajs6f joins08:57
* nbanks joins09:05
* mikeAtUVa joins09:15
* ermadmix joins09:20
<pivotal-bot>Nigel Banks added "Should all Datastreams have DC terms and RELS-EXT associated with them?" https://www.pivotaltracker.com/story/show/6181181209:24
* ksclarke joins09:28
* tecoripa joins09:50
* kaarefc joins09:54
* tecoripa leaves10:02
* ajs6f leaves10:11
* ajs6f joins
escowles: Have you had a chance to look at that indexer PR?
Or branch, rather.
<escowles>not yet -- trying to get %&#$ clustering to work
<ajs6f>np— good luck!10:12
ermadmix: The branch in question is at: https://github.com/futures/fcrepo-jms-indexer-pluggable/tree/ajs6fThrashing10:13
and you can see the new form of IndexerGroupIT there.10:14
* nbanks_ joins10:16
* nbanks leaves10:17
* kaarefc leaves10:19
<ermadmix>ajs6f: Thanks, taking a look now...10:23
<ajs6f>k
ping me for help
<barmintor>ajs6f: how generalized is that offer?10:29
<ajs6f>barmintor: I would cross oceans and climb mountains to bring you a patch.
<barmintor><3
<ajs6f>Besides, I'm waiting on awoods to review my last PR.10:30
<awoods>where does the "besides" come in, ajs6f?10:31
<ajs6f>Meaning that besides my great love and admiration for barmintor, I have nothing better to do.10:32
<cbeer>escowles: isn't clustering fun?
there's so many things that can go wrong, and so little reporting when it does
<escowles>cbeer: i <3 clustering, it's the bestest!
<ajs6f>I hope our users agree.10:33
<escowles>i thought i had it working, but it was just taking forever to timeout and give me an error message...
<barmintor>escowles: I can't get master to start in single-node/mvn jetty:run
I think there's an error
it dies after a couple of minutes10:34
saying it couldn't start the repo factory bean or somesuch
<escowles>barmintor: hmmm... i've been using the minimal config when i do a single-node fcrepo4 and i haven't seen that
but that's the error i got when i tried to start my cluster slave
<barmintor>escowles: what do you do besides mvn jetty:run with heap parms/10:35
<escowles>my standard params are: -server -Dfile.encoding=UTF-8 -Djava.awt.headless=true -Xms512m -Xmx2048m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+DisableExplicitGC
and then for minimal config i do: -Dfcrepo.modeshape.configuration=classpath:/config/minimal/repository.json10:36
<barmintor>man, that's a long MAVEN_OPTS
<escowles>yep, makes for pretty ps listings
<barmintor>I'm not setting the fcrepo.modeshape.configuration proerty, which used to work
<pivotal-bot>Andrew Woods added comment: "This ticket has been overcome by #45629439, and is no longer needed." https://www.pivotaltracker.com/story/show/6115284210:37
Andrew Woods rejected "Fix integration test AtomJMSIT exception related to truncated messages" https://www.pivotaltracker.com/story/show/61152842
<ajs6f>I've been able to run with just heap settings...
but I haven't tried in a while.
<awoods>I am building a fresh master now...10:38
new topic: C4L pre-conference. My understanding was that someone who is actually attending C4L was going to motivate a precon F4 session. It would be great if the volunteer(s) would respond to the fedora-tech email.10:39
<escowles>awoods: i remember the discussion at austin ending with some people saying they were already involved in other preconfs and/or there were already overlapping preconfs that might make it hard to do a f4 preconf10:41
<awoods>escowles: I recall that branch of the conversation as well... now that you bring it up. Is that where we are landing? Or has the state changed?10:43
<barmintor>escowles: specfiying the config property worked- maybe we removed the old default config? or invalidated it somehow? Will look into it.10:44
<pivotal-bot>Chris Beer added comment: "Yes, to DC (although it's debatable whether we should use the DCES or DCTERMS vocabs).
The RELS-EXT vocab i..." https://www.pivotaltracker.com/story/show/61811812
<escowles>i would be willing to help out at a preconf, but i'm not sure about organizing it (got some pushback re: my f4 talk)
barmintor: we talked yesterday about renaming the default profiles -- did anything related to that change?
* nbanks leaves10:46
<awoods>escowles: push back from your local institution?
<escowles>awoods: i forgot to tell declan i was putting in a talk proposal, and he was (understandably) surprised and irritated10:47
<awoods>escowles: sure
<escowles>awoods: so i'm good to give the talk, but submitting a preconf would probably not be a good idea10:48
<barmintor>escowles: I think it's a typo in rest-sessions/repo.json; checking now
<awoods>escowles: That is completely understandable. Is there anyone else right now who would be interested in leading an F4 precon?
<pivotal-bot>Frank Asseg added comment: "I rewrote the whole thing and did not see those exceptions when testing.10:49
I pushed a new branch to https://gi..." https://www.pivotaltracker.com/story/show/61735010
<gregjansen>awoods, escowles: here is what I'm working on re: cluster testing under profiles https://github.com/gregjan/fcrepo-test-profiles/blob/master/pom.xml
<awoods>barmintor: As long as I stop my tomcat F4 webapp, "mvn jetty:run -pl fcrepo-webapp" works for me.
<barmintor>awoods: do you have any env properties set, or JVM related stuff in settings.xml?10:51
<awoods>barmintor: only this: echo $MAVEN_OPTS
-Xmx1024m -XX:MaxPermSize=1024m
<barmintor>awoods: and you're on HEAD of master? Because this has failed consistently for me since yesterday.10:53
<awoods>barmintor: ceae23e7a18b30a9aa283e5a66da290a551fe8ab10:54
<barmintor>hmmm
* nbanks joins10:55
<cbeer>awoods: i'm going to code4lib, but am leading morning and afternoon precons already.10:56
* github-ff joins10:57
[fcrepo4] ajs6f pushed 1 new commit to FixTheHellOutOfJMS: http://git.io/6OXw-Q
fcrepo4/FixTheHellOutOfJMS 139cffb ajs6f: Improved integration test
* github-ff leaves
<awoods>cbeer/escowles/gregjansen: It is sounding like F4 will be pre-conless... last call? anyone?
<ajs6f>escowles: ^^^ Should answer your concern. No counting now. Good advice!
omw to standup. bbs
<barmintor>awoods: If Rob sends me to C4L, I'll do it.
* ajs6f leaves
<awoods>barmintor++, can you ask Rob?10:58
<barmintor>I'll check w/ SPD and Rob later today.
<fasseg>im gonna be a minute late for standup...10:59
* ajs6f joins11:01
<cbeer>ajs6f: you're telling me to merge that?11:04
<ajs6f>cbeer: Yep— take a look and if you like it, roll it.
<cbeer>escowles: ship it.11:05
<pivotal-bot>A. "Horbulaco" Soroka added "Fix indexer gear to use new messaging format" https://www.pivotaltracker.com/story/show/6182050811:06
A. "Horbulaco" Soroka started "Fix indexer gear to use new messaging format" https://www.pivotaltracker.com/story/show/61820508
<awoods>All: comments welcome: http://fedorarepository.org/11:08
<pivotal-bot>A. "Horbulaco" Soroka deleted "Fix indexer gear to use new messaging format" https://www.pivotaltracker.com/story/show/6182050811:10
Andrew Woods edited "jms-indexer-pluggable - comply to header messaging" https://www.pivotaltracker.com/story/show/61750308
* travis-ci joins11:14
[travis-ci] futures/fcrepo4#1320 (FixTheHellOutOfJMS - 139cffb : ajs6f): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/a9be2e5d8bc3...139cffb3bcd2
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/14871389
* travis-ci leaves
* github-ff joins11:16
[fcrepo4] escowles pushed 1 new commit to master: http://git.io/wepDuQ
fcrepo4/master fff0aa7 Esmé Cowles: Merge pull request #178 from futures/FixTheHellOutOfJMS...
* github-ff leaves
<pivotal-bot>Andrew Woods edited "Write recipe for converting sample Fedora 3 content models to Fedora 4" https://www.pivotaltracker.com/story/show/6096348211:19
<ajs6f>ermadmix: The new JMS format has been merged into fcrepo4, so you can start using it.11:21
<bljenkins>Project fcrepo4 build #1477: UNSTABLE in 24 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1477/
<barmintor>well, minimal and single-file configs work. it's the rest-sessions config that won'r start11:22
<ajs6f>ermadmix: Do you have a design for how the indexer can select an appropriate transformation to use to retrieve indexing information, or do you want to have that off to me?
hand that off, I mean
<pivotal-bot>Esme Cowles delivered "Make a JMS serialization that isn't annoying to parse" https://www.pivotaltracker.com/story/show/4562943911:23
<barmintor>mikeAtUVA: I'll just email
<mikeAtUVa>barmintor: thanks, sorry I missed you.
* github-ff joins11:24
[fcrepo4] escowles deleted FixTheHellOutOfJMS at 139cffb: http://git.io/QaPLhQ
* github-ff leaves
<ermadmix>ajs6f: Thanks11:26
<ajs6f>ermadmix: Thanks mean I should take up the design task?
<cbeer>escowles: when you're doing clustering, which configs are you using?
<ermadmix>ajs6f: Is this thrashing branch ready for merge too? https://github.com/futures/fcrepo-jms-indexer-pluggable/tree/ajs6fThrashing/fcrepo-jms-indexer-core11:27
<escowles>cbeer: i was using -Dfcrepo.modeshape.configuration=classpath:config/clustered/repository.json without any ispn config specified
<ajs6f>ermadmix: I think so. You may want to check with escowles, who has been looking at that.11:28
<cbeer>hm. any jgroups config?
<barmintor>Can this be true: the leveldb config works when I'm not running Spotify?
<ajs6f>ermadmix: In fact, he was going to edit it a bit so I wouldn't merge anything without checkin with him first.
<barmintor>whatever.
<ajs6f>barmintor: Is that some kind of locallly-running app? Maybe it uses LevelDB internally?11:29
<escowles>cbeer: yes, i was using variants of the cnfig on the wiki and what we used in austin, e.g.: -Dfcrepo.ispn.numOwners=2 -Djava.net.PreferIPv4Stack=true -Djgroups.tcp.address =0.0.0.0 -Djgroups.tcpping.initial_hosts=132.239.119.14[7800]
<ajs6f>(Port collisions, temp file collisions, that sort of thing?)
<barmintor>ajs6f: maybe!
<cbeer>escowles: do these boxes have multiple interfaces?
<ajs6f>barmintor: Glad we got that settled. :)11:30
<escowles>cbeer: nope, just eth0 and loopback
also got iptables opened up so all TCP traffic between the two machines is allowed
<ermadmix>ajs6f: ok, got to go to meetings, will check in later this afternoon, also which is the design task?
<cbeer>escowles: and no firewall in the mix?11:31
(and, what about UDP?)
<ajs6f>ermadmix: There may not be an issue. I
'll make one.
<escowles>cbeer: i specifically asked for TCP, maybe i need to get UDP opened up too...
<cbeer>escowles: yeah. i think that was part of the problem i was struggling with.. there's some heartbeat code in jgroups that might be doing UDP11:32
<pivotal-bot>A. "Horbulaco" Soroka added "Design configuration for indexer to allow use of fcrepo-transform transformations to produce indexing results." https://www.pivotaltracker.com/story/show/61823412
A. "Horbulaco" Soroka started "Design configuration for indexer to allow use of fcrepo-transform transformations to produce indexing res..." https://www.pivotaltracker.com/story/show/61823412
* travis-ci joins
[travis-ci] futures/fcrepo4#1322 (master - fff0aa7 : Esmé Cowles): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/ceae23e7a18b...fff0aa7a7f4d
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/14872233
* travis-ci leaves
<escowles>cbeer: i did see some "no physical address for XXXXX, discarding message" type of errors, so that would make sense.
<cbeer>could be.11:33
jgroups really doesn't like when you make a partial connection between members
<pivotal-bot>Gregory Jansen added "Bundle test workflow, platform, scale profiles into a cluster-hosted IT based on cargo" https://www.pivotaltracker.com/story/show/6182355411:34
Gregory Jansen edited "Bundle test workflow, platform, scale profiles into a cluster-hosted IT based on cargo" https://www.pivotaltracker.com/story/show/61823554
Gregory Jansen added comment: "cluster host node starts up with chosen profiles.. Benchtool runs (via ant tasks)." https://www.pivotaltracker.com/story/show/6182355411:36
Gregory Jansen added comment: "Next I am looking into a workflow for starting the other nodes in cluster and setting memory footprint fo..." https://www.pivotaltracker.com/story/show/6182355411:37
Gregory Jansen added comment: "https://github.com/gregjan/fcrepo-test-profiles/blob/master/pom.xml
" https://www.pivotaltracker.com/story/show/61823554
Gregory Jansen started "Bundle test workflow, platform, scale profiles into a cluster-hosted IT based on cargo" https://www.pivotaltracker.com/story/show/61823554
Nigel Banks added "Single Click Launcher can't be restarted" https://www.pivotaltracker.com/story/show/61824094
<nbanks>Damn pivotal doesn't seem to work well with Chrome11:39
<cbeer><http://fedora.info/definitions/v4/repository#clusterSize> "4"^^<http://www.w3.org/2001/XMLSchema#string> ;
<escowles>cbeer: i noticed that datatype problem too
<cbeer>escowles: i was just happy the number was 4 :P
<escowles>i was just so happy it had found the other node...
<pivotal-bot>Nigel Banks edited "Single Click Launcher can't be restarted" https://www.pivotaltracker.com/story/show/61824094
Nigel Banks edited "Single Click Launcher can't be restarted" https://www.pivotaltracker.com/story/show/6182409411:40
Nigel Banks added "Single click launcher should allow the user to specify where files should be persisted." https://www.pivotaltracker.com/story/show/6182461211:42
Nigel Banks estimated "Single click launcher should allow the user to specify where files should be persisted." as 2 points https://www.pivotaltracker.com/story/show/6182461211:43
Nigel Banks edited "Single click launcher should allow the user to specify where files should be persisted." https://www.pivotaltracker.com/story/show/61824612
<cbeer>nbanks: are you making separate tickets for all the issues listed in https://www.pivotaltracker.com/story/show/60524458?
<pivotal-bot>feature: Single-click launcher enhancements (unscheduled) / owner:
<nbanks>cbeer: yup
<pivotal-bot>Nigel Banks added "Create a Mac OSX Dock icon for single click launcher" https://www.pivotaltracker.com/story/show/6182473411:44
Nigel Banks edited "Create a Mac OSX Dock icon for single click launcher" https://www.pivotaltracker.com/story/show/61824734
Nigel Banks added "Add optional cmd-line argument to single click launcher to not show the GUI" https://www.pivotaltracker.com/story/show/6182486611:45
Nigel Banks edited "Add optional cmd-line argument to single click launcher to not show the GUI" https://www.pivotaltracker.com/story/show/61824866
Nigel Banks edited "Single click launcher should allow the user to specify where files should be persisted." https://www.pivotaltracker.com/story/show/6182461211:46
Nigel Banks edited "Single Click Launcher can't be restarted" https://www.pivotaltracker.com/story/show/61824094
Nigel Banks edited "Create a Mac OSX Dock icon for single click launcher" https://www.pivotaltracker.com/story/show/61824734
Nigel Banks edited "Add optional cmd-line argument to single click launcher to not show the GUI" https://www.pivotaltracker.com/story/show/61824866
Nigel Banks added comment: "Closing ticket as it's now represented by separate tickets." https://www.pivotaltracker.com/story/show/60524458
Nigel Banks deleted "Single-click launcher enhancements" https://www.pivotaltracker.com/story/show/6052445811:47
* tecoripa joins
<ajs6f>cbeer: Is there any documentation in:11:49
https://wiki.duraspace.org/display/FF/HTTP+API
about the transform endpoints?
Or should I look at the integration tests?
<cbeer>ajs6f: doesn't look like it, huh?
<ajs6f>maybe it's hiding.11:50
* kaarefc joins
<cbeer>ajs6f: guess i cleverly got that change in the interregnum between eddies and awoods
<ajs6f>The brief renaissance of freedom between two awful tyrannies.11:51
<cbeer>ajs6f: given a large enough sample size, round-robin and random selection should be about the same, right?
<ajs6f>Eh? You mean for clustering?
<cbeer>ajs6f: yeah. i'm trying to figure out how lazy i can be11:52
<ajs6f>For our tests, I don't think it will matter much.
Given perfect cluster symmetry.
In the real world, portions of network may behave differently, and we might see that reflected in performance variation.11:53
Or if a particular section of network fails, we might see different failure modes for the cluster based on that selection.
I did not know that fcrepo3 stores all things as XML. Sounds like a good idea. Maybe we should convert all binaries into RDF for fcrepo4?11:56
<escowles>ajs6f: i head fcrepo5 will be written in RDF
* barmintor glares angrily at the room
<ajs6f>escowles: You heard wrong. It's OWL.
And fcrepo6 will be written in pure description logic.11:57
The source code will consist entirely of a lengthy proof of its own existence.
<escowles>ajs6f: right, you prove the possible existence of a perfect repository using OWL, and then your triplestore just makes it so11:58
<ajs6f>That's what we call "inference".
cbeer: Looks like there are two endpoints for fcrepo-transform: fcr:transform and fcr:sparql. Yes?
* escowles goes off to try this with hamburgers
<barmintor>ajs6f: I lied about the leveldb thing earlier. It wasn't spotify, it was accidentally using single-file12:00
* barmintor thinks the world makes a little more sense again
<ajs6f>barmintor: Wait for it. It'll switch back again soon.12:01
<cbeer>ajs6f: fcr:sparql is the sparql search.
(for better or worse.. i thought there'd be more overlap between the sparql query transform and the sparql search codebases)
<ajs6f>cbeer: I was under the (possibly mistaken) impression that we can use fcrepo-transform to run a piece of SPARQL against a single resource to return new RDF, the same way we can run a piece of LDPath against a single resource to create a set of named fields. Is that wrong?12:02
<cbeer>ajs6f: yes, that's true.
<ajs6f>cbeer: And the endpoint which whcih to do that would be fcr:sparql?12:03
<cbeer>no. fcr:transform.
fcr:transform does conneg for ldpath or sparql, i think
<ajs6f>cbeer: Ah, okay, I see. Its all happening in TransformationFactory, which is what confused me.12:04
* ermadmix leaves12:05
* nbanks leaves12:06
* gregjansen points at the hamburgers ^^^
<ajs6f>gregjansen: By pointing at the hamburgers, you have brought them into existence.12:07
<barmintor>ajs6f: he has only brought the URI for the hamburger concept into existence. The dereferenced hamburger, if there is one, has its own unique identifier
<ajs6f>NOT HTTP 14 AGAIN!12:08
<pivotal-bot>Frank Asseg added comment: "Added PR https://github.com/futures/benchtool/pull/7" https://www.pivotaltracker.com/story/show/61735010
* gregjansen points at Wittgenstein
<fasseg>escowles: here's the PR for the benchtool: https://github.com/futures/benchtool/pull/7
<barmintor>DAMN YOU LEVELDB TELL ME WHAT YOU WANT
<bljenkins>Yippie, build fixed!12:09
Project fcrepo4 build #1479: FIXED in 25 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1479/
<cbeer>bash--
<bljenkins>Project fcrepo-fixity-corrupter build #520: SUCCESS in 1 min 41 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/520/12:10
Project fcrepo-kitchen-sink build #690: STILL UNSTABLE in 3 min 14 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/690/12:14
Yippie, build fixed!12:16
Project fcrepo-jms-indexer-pluggable build #311: FIXED in 6 min 57 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-jms-indexer-pluggable/311/
<pivotal-bot>Andrew Woods added "UI glossary documentation" https://www.pivotaltracker.com/story/show/6183027412:28
Andrew Woods edited "UI glossary documentation" https://www.pivotaltracker.com/story/show/61830274
* github-ff joins12:29
[fcrepo4] ajs6f created CodeCleanup (+9 new commits): http://git.io/M6LLhg
fcrepo4/CodeCleanup 1f7e8ce ajs6f: Static imports where appropriate
fcrepo4/CodeCleanup 0fe85ef ajs6f: Improved code syntax for constructors to make more clear the use of JcrTools
fcrepo4/CodeCleanup f8ef698 ajs6f: More whitespace and static import cleanup
* github-ff leaves
* github-ff joins12:31
[fcrepo4] ajs6f opened pull request #181: Code cleanup (master...CodeCleanup) http://git.io/jMUYIA
* github-ff leaves
<pivotal-bot>A. "Horbulaco" Soroka added "Bump Sesame version, also code formatting" https://www.pivotaltracker.com/story/show/61831022
<ajs6f>cbeer: I _think_ I get how fcrepo-transform works now, at least for LDPath. You do a GET against a node, end the path of the node with fcr:transform, and you offer a path _after_ fcr:transform that is a pointer into a special section of repo that contains LDPath transforms. Right?12:32
<cbeer>that contains LDPath transforms for different mixin types (hey, disseminators!)12:33
or, i think, there's an option to POST an LDPath program12:34
?
<ajs6f>Wait, what? So what's the point of the "program" section of the URL?
<cbeer>ajs6f: i could have an LDPath program for doing my solr indexing, and one for my hydra app, and one for who-knows-what12:35
<ajs6f>Right.
How does the "program" path distinguish them?
(If the selection of LDPath is actually eyed off of node types?)
<cbeer>ajs6f: first, select the program type, then find the ldpath program for the node type.12:36
(unless it's the other way around.. let me get the code in front of me)
<ajs6f>So, I can use program path to select a piece of the directory of LDPath programs, then node type is used within that section?12:37
* ermadmix joins
<ajs6f>That's kind of what LDPathTransform looks like, to me.
<cbeer>yeah, that looks right
<pivotal-bot>Mike Durbin started "Add alternate versioning policy." https://www.pivotaltracker.com/story/show/61421690
Mike Durbin added comment: "Updated PR." https://www.pivotaltracker.com/story/show/6142169012:38
Mike Durbin finished "Add alternate versioning policy." https://www.pivotaltracker.com/story/show/61421690
* github-ff joins
[fcrepo4] ajs6f created CodeCleanup2 from CodeCleanup (+0 new commits): http://git.io/orS8zg
* github-ff leaves
* github-ff joins12:39
[fcrepo4] ajs6f closed pull request #181: Code cleanup (master...CodeCleanup) http://git.io/jMUYIA
* github-ff leaves
<ajs6f>Okay. I think it may be easier (for this indexer stuff) to start by using the POST endpoint, where the indexer would maintain control over the transformations...
* github-ff joins12:40
[fcrepo4] ajs6f opened pull request #182: Code cleanup2 (master...CodeCleanup2) http://git.io/3oaI5A
* github-ff leaves
* github-ff joins
[fcrepo4] ajs6f deleted CodeCleanup2 at 144951d: http://git.io/VBFnpA
* github-ff leaves
<cbeer>ajs6f, mikeAtUVa: here's a weird error with clustering and versioning on..12:43
https://gist.github.com/cbeer/0dc8ae44ce248eac397f
this is on a request to create /bb/072/gy/6494 in the first place
<ajs6f>From benchtool?
<cbeer>no. my stanford data ingest.12:44
<pivotal-bot>A. "Horbulaco" Soroka edited "Code formatting" https://www.pivotaltracker.com/story/show/61831022
<ajs6f>Problem with threading? Did you try to alter a property too quickly?
<tecoripa>@fasseg: you can now pass a username and password for fcrepo4 tests in benchtool, too?12:48
<barmintor>ajs6f: Am I correct in assuming that post merge of FixTheHellOutOfJMS we no longer need to have SimpleObserver configured/12:49
<ajs6f>barmintor: Incorrect. SimpleObserver hasn't changed in the least. All of the changes in FixTheHellOutOfJMS occurred in fcrepo-jms.12:51
<barmintor>damn
<ajs6f>Why?
<barmintor>b/c the bazillions of node lookups in DefaultFilter are a drag on performance12:52
<ajs6f>We need to think harder about how we tell what is and what isn't a "fedora" resource.
* ermadmix_ joins
<ajs6f>Doing it by mixin is powerful but expensive.
<barmintor>ajs6f: Maybe, but that seems mostly orthogonal to "JCR Event listeners ought not to look the nodes back up"12:53
<ajs6f>It is highly relevant. The reason those lookups occur is so that we can filter only to events concerning "Fedora" resources, and the only way to do that is to look them up and get their mixins. that's where those ops are coming from.12:54
<barmintor>right, that seems symptomatic to me
<ajs6f>I agree, but we do need to filter all the persistence and other events.12:55
<awoods>gregjansen/mikeAtUVa: Can we chat @4pm?
<barmintor>Well, I'm not sure about that
<ajs6f>We need some highly-efficient way to find out what is and isn't a "Fedora" resource.
* ermadmix leaves
<ajs6f>barmintor: When we didn't, we got dozens or hundreds of events for the creation of a single Fedora resoure.
That's why we introduced the whole filtering thing to begin with.
<barmintor>We only need to do that if our requirements are 1) Reproduce FCR3's messages and 2) use a JCR event listener to do it
<ajs6f>Not at all. Our requirement is "Don't spooge hundreds of useless JCR system events at people trying to drive workflow fom the event stream."12:56
<bljenkins>Project fcrepo-fixity-corrupter build #521: SUCCESS in 2 min 5 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/521/
<barmintor>we could have, for example, fcrepo-kernel emit fcrepo event messages
<ajs6f>it does now.
<barmintor>if fcrepo-kernel sends the messages, then simpleobserver does nothing12:57
<ajs6f>simpleobserverr is _in_ kernel
It's what is doing it.
jms is driven by simpleobserver
<barmintor>I reform my claim: NodeService/DatastreamService/ObjectService could emit messages
or the http-commons layer could12:58
<ajs6f>Yes, then we can't use the JCR events to drive the whole process. We can do that. We'd have to write the code to do it. (And establish a message form and so forth.) But we could do that.
<barmintor>Right, this is what I am saying
<ajs6f>Not http-commons. It's not an http concern. It's a kernel concner.
<barmintor>In this specific case, I think there's an argument that it's a fcrepo-http-legacy concern12:59
* nbanks joins
<barmintor>but that's a narrow view
<ajs6f>Oh, no. It's not legacy. The mesage format on JMS is legacy, but not the _idea_ of messaging. :)13:00
That's a pretty narrow view.
* barmintor is narrow minded, with narrow eyes, and narrow views
<ajs6f>Do you want to try and write something using Guava's event bus that is driven by the *Service classes? It wold mean ensureing that all action goes through those classes (and not, say, the JCR APi). I don't like that very much at all.13:01
I'd like to be able to use the JCR APi when I want, and the *Service classes when I want.
<barmintor>~400 ds node creations translate into ~27k Repository.login's & node lookups
<ajs6f>Which is why I suggested ^^^ finding a new way to distinguish Fedora resources, which would solve the problem.13:02
<barmintor>with data in the event?
<ajs6f>What data?13:03
<barmintor>the event info
<bljenkins>Project fcrepo-jms-indexer-pluggable build #312: UNSTABLE in 9 min 5 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-jms-indexer-pluggable/312/
<ajs6f>If we can use the data in the JCR event to make decisions about whether to publish it as a Fedora event or not, we don't _need_ to look up the node.
<barmintor>so true, it hurts
<ajs6f>Just as we no longer need to look up the node in the JMS stuff.13:04
* barmintor winces
<ajs6f>Eh?
Did you eat a bad olive?
<barmintor>ajs6f:it was illustrative wincing. How can we jam data into the event info?13:05
<ajs6f>ObservationManager.setUserData(String userData)13:06
<barmintor>No.
<ajs6f>http://www.day.com/specs/jcr/2.0/12_Observation.html#12.3.5 User Data13:07
Why not?
More importantly, what do you want to set?
If we use what JCR gives us, we avoid the problem.
<barmintor>Because that just moves the node lookups out of the filter into an observation manager
<ajs6f>No, it doesn't. It does not.13:08
The use data is associated to the Session, not any node.
<barmintor>exactly
<ajs6f>I'd still like to know what you want to add to the event?
* ermadmix leaves13:09
<ajs6f>I'm saying that if we want to be efficient, we should not be _trying_ to add anything to the event. We should use what's there.
<barmintor>ajs6f: a flag that indicates whether it's a "fedora" event or not
what's there is inadequate13:10
<ajs6f>I think the adequacy of what's there depends on our pathing.
<barmintor>!!!
<ajs6f>Wait.
Wait.
* barmintor waits13:11
<ajs6f>Look at ObservationManager.addEventListener()
<barmintor>yes
<ajs6f>It lets you offer a series of NodeTypes on which to filter. Isn't that what we want to do?
<barmintor>no13:12
<ajs6f>hy not?
Thats what we do now.
<barmintor>wait, nodeTypes or eventTypes?
<ajs6f>nodetypes and/or eventtypes
both
http://www.day.com/specs/jcr/2.0/12_Observation.html#12.5.2 Adding an Event Listener13:13
<barmintor>ok, we're using a null parm there. let me try doing that with proper types13:14
<ajs6f>It all depends on MODE. If they impl'd that really well, it could be much more efficient than us looking up the node. If they didn't, it may be no better. But if it isn't, we might offer a patch.13:15
<cbeer>mikeAtUVa: did the optional versioning stuff land in master already?13:17
<gregjansen>awoods/mikeAtUVa: I have a meeting from 3-4:30 today..13:19
<cbeer>mikeAtUVa: oh, that's in https://github.com/futures/fcrepo4/pull/174, right?
<pivotal-bot>Esme Cowles added comment: "I've added my F3 benchmarks to the wiki: ""13:20
https://wiki.duraspace.org/display/FF/Single-Node+Performance+Testing" https://www.pivotaltracker.com/story/show/60557024
<awoods>gregjansen/mikeAtUVa: how does 4:30pm sound?
<pivotal-bot>Esme Cowles finished "Single-node F3 Ingest Benchmarks" https://www.pivotaltracker.com/story/show/60557024
<gregjansen>awoods: 4:30 works13:22
<awoods>mikeAtUVa?
<pivotal-bot>Esme Cowles added comment: "I have updated the Single Node Benchmark results: ""13:23
https://wiki.duraspace.org/display/FF/Single-Node+Perform..." https://www.pivotaltracker.com/story/show/60556948
<bljenkins>Project fcrepo-fixity-corrupter build #522: SUCCESS in 1 min 9 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/522/13:24
<pivotal-bot>Esme Cowles added "Test 2-Node Ingest on UCSD cluster" https://www.pivotaltracker.com/story/show/61835946
<bljenkins>Project fcrepo-kitchen-sink build #691: STILL UNSTABLE in 3 min 22 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/691/13:27
* github-ff joins13:28
[fcrepo4] escowles pushed 1 new commit to master: http://git.io/0eJaxg
fcrepo4/master ef7ce03 Esmé Cowles: Merge pull request #177 from futures/http-api-fixes...
* github-ff leaves
<pivotal-bot>Esme Cowles delivered "Ensure HTML AJAX operations display all errors" https://www.pivotaltracker.com/story/show/61753458
* github-ff joins
[fcrepo4] escowles deleted http-api-fixes at ac50c5c: http://git.io/wc-B4g
* github-ff leaves
<pivotal-bot>Chris Beer accepted "Ensure HTML AJAX operations display all errors" https://www.pivotaltracker.com/story/show/6175345813:30
Chris Beer accepted "Make a JMS serialization that isn't annoying to parse" https://www.pivotaltracker.com/story/show/45629439
* ermadmix joins13:31
<pivotal-bot>Esme Cowles added "Improve AJAX Error Handling of Total Repo Failure" https://www.pivotaltracker.com/story/show/6183661213:32
<mikeAtUVa>awoods: 4:30's fine... sorry I was away at lunch13:36
<awoods>gregjansen/mikeAtUVa: thanks. 4:30 it is. standard g-hangout.13:37
<mikeAtUVa>cbeer: no, it hasn't been merged yet, you're right it's PR 174
<pivotal-bot>Chris Beer estimated "Improve AJAX Error Handling of Total Repo Failure" as 1 point https://www.pivotaltracker.com/story/show/6183661213:38
<gregjansen>awoods: great! Are the fcrepo snapshots in a public repo?13:41
<awoods>gregjansen: yes13:42
gregjansen: https://oss.sonatype.org/index.html#welcome
<gregjansen>awoods: thanks!
<cbeer>mikeAtUVa: ok. i think i'm going to wait on this clustering stuff until that gets merged. seems easier than trying to figure out what's actually wrong.
<awoods>gregjansen: That repo is in our sonatype parent pom.
* travis-ci joins13:43
[travis-ci] futures/fcrepo4#1323 (master - ef7ce03 : Esmé Cowles): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/fff0aa7a7f4d...ef7ce03ec1d7
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/14879509
* travis-ci leaves
<bljenkins>Project fcrepo-fixity-corrupter build #523: SUCCESS in 1 min 6 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-fixity-corrupter/523/13:48
* github-ff joins
[fcrepo4] escowles closed pull request #180: Update admin SPARQL search to support rdf:type assertions about mixins (master...sparql-rdf-type) http://git.io/_7kFaA
* github-ff leaves
<pivotal-bot>Esme Cowles added comment: "Fixed by https://github.com/futures/fcrepo4/pull/180" https://www.pivotaltracker.com/story/show/60962506
Esme Cowles delivered "Update administrative search to support rdf:type translations" https://www.pivotaltracker.com/story/show/60962506
Chris Beer accepted "Update administrative search to support rdf:type translations" https://www.pivotaltracker.com/story/show/6096250613:49
<bljenkins>Project fcrepo-kitchen-sink build #692: STILL UNSTABLE in 5 min 4 sec: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/692/13:53
<barmintor>ajs6f: ok, working on a branch that adds a method to the EventFilter interface, so that the wrapped filter tells the ObsMgr what nodeTypes it is relevant to13:56
<ajs6f>Did it seem to help with the crazy explosion of lookups?13:57
<fasseg>escowles: I closed the benchtool PR, i just realized that it needs a change...
<barmintor>dont know yet, have to update IT
<ajs6f>Fair enough. Fingers crossed.
<escowles>fasseg: ok, i just saw that -- i'm testing how large a file i can ingest/serve right now, but i can review the benchtool PR whenever you're ready
<pivotal-bot>A. "Horbulaco" Soroka added "Pack ResultSetStreamingOutput in a MessageBodyWriter<ResultSet>" https://www.pivotaltracker.com/story/show/6183899013:59
Esme Cowles started "Determine filesize ingest limit via REST API" https://www.pivotaltracker.com/story/show/61773430
<barmintor>Aargh!14:04
I think we won't get node creation messages this way, because the node doesn't have the right type when it's created14:07
and fedora:object & fedora:datastream are only mixin types14:08
Hmm. maybe.14:11
<bljenkins>Project fcrepo4 build #1483: UNSTABLE in 22 min: http://ci.fcrepo.org/jenkins/job/fcrepo4/1483/
Chris Beer: Update admin SPARQL search to support rdf:type assertions about mixins
<barmintor>those are supposed to be session-write methods
<ajs6f>Which methods?
<barmintor>findOrCreateNode14:13
<ajs6f>on jcrTools?
<barmintor>the underlying methods t them
<ajs6f>Yeah, the JCR should be able to create the nodes _with_ mixins. (Thus firing the events...)14:14
<barmintor>ajs6f: I agree, though that isn't happening (yet)
<pivotal-bot>Andrew Woods started "NOT WORKED: Fix integration test AtomJMSIT exception related to truncated messages" https://www.pivotaltracker.com/story/show/61152842
Andrew Woods accepted "NOT WORKED: Fix integration test AtomJMSIT exception related to truncated messages" https://www.pivotaltracker.com/story/show/61152842
<barmintor>ajs6f: org.junit.ComparisonFailure: Got wrong method! expected:<[inges]t> but was:<[modifyObjec]t>14:15
<ajs6f>barmintor: If you think it might be because of JcrTools, I can write something to do it with the JCr API directly, if that wold help.
barmintor: That sounds suspiciously like the mixins are being added "after the fact".
<barmintor>indeed14:16
* github-ff joins
[fcrepo4] ajs6f created FormattingInFcrepoTransform (+1 new commit): http://git.io/oU0j9w
fcrepo4/FormattingInFcrepoTransform 3790f9c ajs6f: Pure code reformatting
* github-ff leaves
* github-ff joins14:17
[fcrepo4] ajs6f opened pull request #183: Pure code reformatting (master...FormattingInFcrepoTransform) http://git.io/mX5gcA
* github-ff leaves
<pivotal-bot>A. "Horbulaco" Soroka added comment: "https://github.com/futures/fcrepo4/pull/183" https://www.pivotaltracker.com/story/show/61831022
<barmintor>or, that the test only expects a single message, and it's getting 2
<pivotal-bot>A. "Horbulaco" Soroka finished "Code formatting" https://www.pivotaltracker.com/story/show/61831022
<ajs6f>barmintor: Which test? With some good advice from escowles, I mamaged to get rid of all the counting in the JMS tests. We can do that elsewhere.
<barmintor>AtomJMSIT and HeadersJMSIT14:18
<ajs6f>Oh, there's no counting in HeadersJMSIT
But AtomJMSIT probably still has it.
I didn't go and fool with that.
How is HeadersJMSIT filing?
or failing, if that sounds better.14:19
<barmintor>"Found no message with correct identifer and correct event type!"14:21
waitaminutewaitaminutewaitaminute14:26
these damned tests
<ajs6f>Now is that a helpful message or what I'm rally proud of that one.
I guess "Found no message with both correct identifier and correct event type!" might have been more clear.14:27
<barmintor>every test in this class makes use of findOrCreate("test1")
but no one deletes it
<ajs6f>In HeadersJMSIT?
It shouldn't.
<barmintor>AtomJMSIT
<ajs6f>Oh, oh.
k
afk bbs
* travis-ci joins14:31
[travis-ci] futures/fcrepo4#1325 (FormattingInFcrepoTransform - 3790f9c : ajs6f): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/commit/3790f9ca2ba7
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/14882078
* travis-ci leaves
<nbanks>ajs6f & barmintor: I was just reading though your conversation, and I was wondering why aren't fedora objects / datastreams primary types? If that would solve the problem of having the right type at creation?
<barmintor>nbanks: A fine question, that I leave to sharper intellects. ajs6f & cbeer, namely.14:32
* ajs6f leaves
* ajs6f joins
nbanks: As far as I know (could be wrong) you can create a node with mixins (at creation time) so it shouldn't be a problem. In another aspect, when we started building the type system, we wanted the Fedora view of a JCR repo to be something that can be overlaid over various possible underlying JCR node type systems. For example, we work a lot with nt:file and nt:folder directly, but there's no reason someone couldn't introduce primary types that inherit 14:35
But that's where we were in our heads in the spring.14:36
* ermadmix leaves
<ajs6f>So many things have changed...
leaves have fallen and there's a bite in the air...
perhaps it's time to set aside those youthful dreams?
<barmintor>don't stop believing, ajs6f. Hold on to that feeling.14:37
<ajs6f>My faith in types systems springs eternal.14:38
<nbanks>ajs6f: haha, so If I'm understanding correctly it was so clients which support the 33 default JCR node types would still be able to function correctly.
<ajs6f>It was so that clients who want to add _their_ action to the JCR-spec types would easily mixin Fedora types.
Hence… mixins.
But all of this may not matter.14:39
the communities may be not have much in common.
JCR users may not care at all about Fedora, by and large, and Fedora users may want JCR hidden, by and large.
In which case, and if it makes impl much easier, we could go to primary types.
barmintor: Thoughts on the awesomeness of those tests?14:40
<barmintor>ajs6f: still chewing
<ajs6f>k. ping me if I can help.14:41
<nbanks>ajs6f: I see, thanks!
<ajs6f>nbanks: You understand all this? Great! Please document it in the wiki, because I have no idea how this stuff is supposed to work. I'll read it when I get a chance.
<barmintor>*gulps*14:43
I wonder if this is because you're supposed to check eventType with bitmasks?14:44
<nbanks>ajs6f: I only partially get it. I understand the justification, but is there really a community out there that wants to mix fedora concepts into their existing node types, aside from people who actually use fedora directly?
<ajs6f>nbanks: Like I said, we were young and full of beans.14:45
Now, my grey hairs tell a sad tale of broken dreams and disappointment.
<nbanks>ajs6f: Shall we try primary types then?14:46
<ajs6f>Well, let's see whether barmintor can slay the dragon using only his Sword of Mixin +2. If not, we'll break out the Wand of Primary Types.14:47
<tecoripa>nbanks: the only community I could think of that may conceivably want to do the mixins with existing node types are people who already have CMIS repositories. All five of them.
<ajs6f>Is anyone else having problems getting to the Duraspace wiki?
<tecoripa>mikeAtUVa can tell you more about that.
<barmintor>I just want to get this change working and profiled before I go explain to my manager why I've done jack-all for Columbia the last two weeks14:48
<ajs6f>Well, what has Columbia done for you lately?
<barmintor>then at least I can point to performance metrics
<ajs6f>If you get this thing working, then we should put off any discussion about moving to primary types for a bit, for the simple reason that it would involve a lot of code changes and we haven't got a good reason it needs to happen before this release.14:49
<barmintor>+1
* gregjansen leaves14:55
* ermadmix joins15:06
* nbanks leaves15:15
* tecoripa leaves15:17
<ajs6f>I've added some notes to:15:19
https://wiki.duraspace.org/display/FF/Design+-+Customizable+Search+Index
about how we could map into sets of named fields (aka Solr docs).
Comments welcome!
<fasseg>escowles: I reopened the PR so you can check it out. I'm gonna be back later but I'll have a break now...15:25
<escowles>ok, i'll take a look
<fasseg>btw I only get comparable performance between fcrepo3 and 4 using the minimal config...15:26
with every other config it does not look good for fcrepo4 on my local box using the updated tests..15:27
single-config was 2 orders slower than fcrepo3 default config :/15:28
read performance seems much better though
<ajs6f>cbeer: I think we've talked about this before, but maybe not. LDPath has an extension module for Freemarker that allows people to use LDPath inside Freemarker. That might make our HTML templates much more readable. Did we think about this and decide not to for some reason?15:29
<fasseg>220mb/sec for fcrepo4 compared to 170mb/sec from fcrepo3
<barmintor>fasseg: I'm pretty sure we will see an uptick in ingest performance when we get this event/JMS thing sorted out15:30
<ajs6f>fasseg: Let me say that for you: "We've seen fcrepo4 perform in single-node configuration with tens of mb/sec better throughput than fcrepo3." See how much better that sounds?
<barmintor>ajs6f: remind me, when this part is fixed, that the LegacyEventFactory *also* looks nodes up15:31
<ajs6f>ermadmix: Doing okay with reworking the indexer to use the new message format?
<ermadmix>ajs6f: Read design notes changes, looks like that touches on all of it.
<barmintor>but that's a much smaler slice of static
<ajs6f>barmintor: Cool. I think we have to decide just how far we're willing to go to support the fcrepo3 message format.15:32
ermadmix: Thanks1
<fasseg>barmintor: that'd be great, and I was using a build from the day before yesterday so you might already see sth better
<ermadmix>ajs6f: Re: reworking the indexer, should I wait for the latest to be merged (ajs6fThrashing)? (I ask as I head to meeting, will be back at 4:30)15:35
<ajs6f>ermadmix: I said yes earlier, because I thought that merge would happen this morning.15:36
escowles: Can I help that happen more quickly? ermadmix is waiting on it...
<escowles>ajs6f: sorry, i haven't gotten to that yet -- can you merge the TestIndexer into IndexerGroupIT like we talked about? i can review it quickly once that's done15:40
<ajs6f>escowles: Yes, I will do that now.
<escowles>ajs6f: though won't it be hard to test the indexer before it's updated to handle the new JMS message format?15:46
<ajs6f>escowles: yes, the tests still fail. The idea here is that they will start passing when the update (new format and therefore delete messages) comes through.15:47
escowles: almost done with the change.
escowles: I take it back. This turns out to be harder than I thought, because of the Spring entanglements. Ironically, It's almost the same problem we ran into a few days ago: having to keep the context of the beans under test and the context of the test beans separate. Let's not wait on this, because the code isn't bad as it stands.16:01
Push it, if you have no objection.
<escowles>ajs6f: i'll click the big green button!16:02
<ajs6f>WE HAVE A BIG GREEN BUTTON?!?
Why did no one tell me about this?
Every repository should come equipped with a BGB.
afk bbs16:03
* ajs6f leaves
<escowles>um, i guess i'll have to make a pull request in order to get a big green button...
* ajs6f joins16:05
<cbeer>ajs6f: yup, we talked about it. and then decided it was a bunch of work for not too much user-facing changes16:07
ajs6f: and most of the time we just wanted to dump a bunch of triples anyway
<ajs6f>cbeer: Cool. Just wanted to make sure we were forgetting something.16:09
<cbeer>ajs6f: yup. we're forgetting it good.
<ajs6f>Right. That's what I meant, or said, or thought, or possibly wrote down, pushed into a bottle and threw into the sea.16:10
* ermadmix leaves16:11
<pivotal-bot>Esme Cowles added comment: "Resolved with https://github.com/futures/benchtool/commit/e98090c2b168835e4907a017483f5059a12f96f9" https://www.pivotaltracker.com/story/show/6173501016:13
Esme Cowles delivered "Clean up bench-tool" https://www.pivotaltracker.com/story/show/61735010
* github-ff joins16:16
[fcrepo-jms-indexer-pluggable] escowles pushed 1 new commit to master: http://git.io/g11ZmQ
fcrepo-jms-indexer-pluggable/master 64812b9 Esmé Cowles: Non-functional update that makes tests less brittle, introducing a new TestIndexer class that allows ITs to check whether messages have been received without having to count files/triples.
* github-ff leaves
* github-ff joins16:17
[fcrepo-jms-indexer-pluggable] escowles closed pull request #14: Making tests that check for presence of triples less brittle (master...test-query-bugfix2) http://git.io/UEbKcA
* github-ff leaves
<ajs6f>escowles/ermadmix: I just remember that I forgot to add a timeout for that IndexerGroupIT's tests.16:18
It's no big deal right now.
ermadmix: Don't let it stop you.
<pivotal-bot>Andrew Woods added "Integrate AuthZ API into F4 HTTP API documentation" https://www.pivotaltracker.com/story/show/61850274
Andrew Woods edited "Integrate AuthZ API into F4 HTTP API documentation" https://www.pivotaltracker.com/story/show/6185027416:19
<ajs6f>But we should fix it at some point. I'll do that, or ermadmix can do it as part of his immediate work, or whatever
<escowles>ajs6f: i just merged the ajs6fThrashing branch as-is, so i think it'll wait for ermadmix16:20
<pivotal-bot>A. "Horbulaco" Soroka added "Documentation for fcrepo-transform's HTTP endpoints." https://www.pivotaltracker.com/story/show/61850448
<ajs6f>escowles: +1
There's been enough confusion already.
<pivotal-bot>Andrew Woods added "Expose AuthZ Service/API at Root of F4 webapp" https://www.pivotaltracker.com/story/show/6185048216:21
Andrew Woods edited "Expose AuthZ Service/API at Root of F4 webapp" https://www.pivotaltracker.com/story/show/61850482
* tecoripa joins16:22
@escowles: the new-and-improved benchtool is ready to go?16:23
<escowles>tecoripa: yep, just merged it a few mins ago
<tecoripa>@escowles: great. I'll re-run my tests (shouldn't be much difference, I hope), and then do the authz tests.16:24
* ajs6f leaves16:26
<tecoripa>escowles: have you been pulling in changes to fcrepo4 when re-running your tests? My last git pull on that project was before thanksgiving.
* travis-ci joins16:27
[travis-ci] futures/fcrepo-jms-indexer-pluggable#36 (master - 64812b9 : Esmé Cowles): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo-jms-indexer-pluggable/compare/b7660d43b772...64812b96e493
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo-jms-indexer-pluggable/builds/14888625
* travis-ci leaves
* ajs6f joins16:28
<pivotal-bot>Chris Beer added comment: "Check out e.g. https://github.com/futures/fcrepo4/blob/master/fcrepo-webhooks/src/main/java/org/fcrepo/webhoo..." https://www.pivotaltracker.com/story/show/6185048216:30
Chris Beer edited "Expose AuthZ Service/API at Root of F4 webapp" https://www.pivotaltracker.com/story/show/61850482
<escowles>tecoripa: yes, i've been pulling changes for fcrepo4 and i've started putting the git commit id for which version i'm testing in my result matrix: https://wiki.duraspace.org/display/FF/Single-Node+Performance+Testing#Single-NodePerformanceTesting-CombinedPerformanceTuningMatrix
<tecoripa>sounds good. I'll do the same.16:31
I like your tables better, too.
<ajs6f>ermadmix: The impl'ing of the index mapping design should take place in IndexerGroup, but that's also where the messages are received and parsed, so I don't think we want to work on that at the same time. I'm going to defer to you changing the expected format of the messages and then come in behind you. On a related note, I'd suggest that you break out the parsing of the messages into a separate class, so as to separate the concerns of message reception an16:34
<escowles>ajs6f: you know ermadmix isn't back yet, right?16:35
* ermadmix joins16:47
* tecoripa leaves16:53
<ajs6f>HE'S BACK NOW~
ermadmix: ^^^
* tecoripa joins17:03
tyring to build the latest fcrepo on my linux workstation
and I keep running into a failed test:
<ajs6f>We don't support Linux. Only AmigoDOS.
<tecoripa>posting gist now:
Running org.fcrepo.integration.JQLConverterIT17:04
Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.433 sec <<< FAILURE! - in org.fcrepo.integration.JQLConverterIT
ajs6f: not even a Commodore? to the brig with ye
<escowles>ajs6f/tecoripa: fcrepo4 HEAD builds for me on MacOSX, but I'm seeing the same issue on Linux...17:05
<tecoripa>https://gist.github.com/sprater/7778292
<ajs6f>Where is JQLConverter?17:06
fcrepo-transform?
<escowles>ajs6f: yep
<ajs6f>cbeer?
(That's his baby.)17:07
<escowles>i imagine this is related to the rdf:type sparql update i merged earlier today
perhaps the power of the big green buttons is too big for me to wield after all...
<ajs6f>Meaning resources are now coming back with triples that confuse fcrepo-transform?
That would make sense.17:08
<tecoripa>I haven't parsed through the test result yet, but I'm wondering if it isn't a simple ordering assertion error... like sometimes happens if your arrays don't come back in the order you expected
<ajs6f>well, I'm out of here for the day. I'm glad I could help you, tecoripa.
* ajs6f leaves
* tecoripa vows to return the favor.17:09
<escowles>tecoripa: that could be, or maybe some other commit has changed the format of the response...
<ermadmix>ajs6f: ??? ... at least I see you "better" merged
<escowles>ermadmix: see the last commit in fcrepo-jms-indexer-pluggable: https://github.com/futures/fcrepo-jms-indexer-pluggable/commit/64812b96e49377c6cadcf7c444a58d81b44f916d17:11
the tests don't pass, but that has a new class called TestIndexer that should make it easier to have robust ITs that don't fail if you get a different number of triples/files/etc.
<tecoripa>escowles: I see the difference; it may be related to extra data being added for the test, messing up offsets:17:13
[ns00[1:bb3652744n] is expected (notice "1:")
[ns00[3:bb3652744n] is returned (notice "3:")17:14
<escowles>so maybe it's a namespace issue?
ns001 v. ns003?
<tecoripa>yes, that seems to be it
ns001:bb2765355h is the object expected, but ns003:bb2765355h is returned...17:15
and so on.
<escowles>that sounds relatively easy to fix, but doesn't explain why it's passing on MacOSX but failing on Linux17:16
<tecoripa>yeah, that's what has me scratching my head, too17:17
(that, and those pesky lice)
escowles: I need to head home now, but shoot me an email if you'd like me to test later on, if you or cbeer merge in a fix17:19
or if you'd like a full dump of the output17:20
<escowles>tecoripa: probably won't hear from me -- i'm heading out in a few mins too, but i'll see if cbeer can take a look at this...
<tecoripa>ok, I'll be around later on
<cbeer>tecoripa: is that really with a clean store?
<tecoripa>cbeer: yes. MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=1024m" mvn clean install17:21
<cbeer>are you sure this isn't working against $TMPDIR in any way?
<tecoripa>ah, good question
<cbeer>because what that says to me is you have registered 2 new namespaces before that test ran
and, yes, we probably should have a more robust test that resolves namespace prefixes or something17:22
* github-ff joins17:23
[fcrepo4] mikedurbin opened pull request #184: Added Velocity template for HTML view of fcr:versions. (master...versioning-4) http://git.io/mPSM3g
* github-ff leaves
<pivotal-bot>Mike Durbin added comment: "https://github.com/futures/fcrepo4/pull/184" https://www.pivotaltracker.com/story/show/61474236
Mike Durbin finished "Write a velocity template for the fcr:versions endpoint." https://www.pivotaltracker.com/story/show/61474236
<tecoripa>trying it now, after deleting /tmp/modeshape-binary-store
maybe the test could empty out the repo before running?17:24
or wipe it away and start afresh?
<cbeer>or not write to $TMPDIR, if that's really what's happening17:26
<tecoripa>well, something in the test suite is creating /tmp/modeshape-binary-store17:27
maybe set the system property java.io.tmpdir to target/data, or something like that, in the pom, for tests17:28
same error. it looks like an earlier test in the integration suite creates the store, but doesn't clean up after itself.17:30
ok, I'm outta here.17:31
* tecoripa leaves
<barmintor>ok, not sure of details, but apparently sometimes an observer of a specific nodeType doesn't get creation events of the type is a mixin18:07
* kaarefc leaves
<barmintor>but it *does* get a propertyAdded event for jcr:created18:08
so I'm just going to infer the creation from that in SimpleObserver
while I try to figure out why it doesn't always get the other event
* ermadmix leaves18:30
* escowles leaves18:52
* ksclarke leaves18:56
<pivotal-bot>Andrew Woods edited "Expose AuthZ Service/API at Root of F4 webapp" https://www.pivotaltracker.com/story/show/6185048220:22
* ksclarke joins20:24
<bljenkins>Project fcrepo-kitchen-sink build #693: STILL UNSTABLE in 10 min: http://ci.fcrepo.org/jenkins/job/fcrepo-kitchen-sink/693/23:30

Generated by Sualtam