Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* kaarefc joins03:14
* kaarefc leaves
* kaarefc joins03:17
* kaarefc leaves03:18
* nbanks joins03:28
<pivotal-bot____>Edwin Shin finished "Add escowles, ap2972, and nbanks to shared calendar" https://www.pivotaltracker.com/story/show/4752989306:55
Edwin Shin edited "Create cluster.fcrepo.org as a CNAME for futures6" https://www.pivotaltracker.com/story/show/4766641106:56
* VincentNG joins07:21
* escowles joins09:09
* nbanks leaves10:10
* ajs6f joins10:33
<barmintor>eddies, et al: I'm using my time in-between meetings/interviews to try to figure out the IT problems for the bagit conector10:35
<ajs6f>barmintor: the IT I wrote at first?10:36
it's more of a system test, actually, but w/e it should work
<ajs6f>Yes, it is a system test.
It seems like the repo isn't going to the fed connector. I put logging in that isn't getting executed.10:37
<barmintor>yeah. the connector gets started, but it doesn't get queried
<ajs6f>But the jSON config is virtually the same as the example (w/ FileSystemConnector) that _does_ work.
<barmintor>yes: weird.
<ajs6f>I haven't yet tried turning on TRACE logging for MODE, but cbeer has had good results from that.10:40
* nbanks joins10:48
* ajs6f leaves10:53
* ajs6f joins10:58
<eddies>ajs6f, cbeer: standup11:01
<eddies>the same :P
<ajs6f>which is?11:02
bookmark it, eh?
<eddies>he's out interviewing today, i thought
<barmintor>Ahh… errr… we must be fast.
<eddies>altho, if he can make standup, great
<barmintor>ajs6f: "No node exists at path '/objects/BagItFed1' in workspace 'fedora'"11:06
so there you go
It's not getting to the connector.
The connector logs when it's asked for something. But I never saw it get asked.
<barmintor>I'm suspicious of the projection config11:07
<ajs6f>Fair enough. That does seem like the prime suspect.
<barmintor>which doesn't *appear* to be tested in the MODE tests
^^ programmatic projection config ^^
<ajs6f>Well, the sample config I gist-ed did work.
Does work.
But I haven't traced through to see _how_.11:09
<pivotal-bot____>Chris Beer added "Add new people to the google docs" https://www.pivotaltracker.com/story/show/4772678711:10
Chris Beer edited "Add new people to the google docs" https://www.pivotaltracker.com/story/show/47726787
<VincentNG>eddies: i heard back from Robert Swain and budget is tight so they won't be sending me to Hackfest11:15
<barmintor>ajs6f: the projected node at /objects exists11:20
but has no children11:21
<ajs6f>But isn't that set up by the modeShapeSpringBeanRepoFactoryHgueLongNameClass?
<barmintor>Err… if it is, there's probably some trouble with trying to map it to a root projection11:22
But why did the gist-ed config work, then?
<barmintor>ok, let's test that by changing the projected node
Have you turned on TRACE?
<ajs6f>Hm. Any info about federation coming into play at all?
Is it even mentioned?
<barmintor>No. Just the init of the connector11:24
<barmintor>what is this gisted config you were talking about
<ajs6f>I'm starting to wonder if our connector doesn't do _something_ that the FileSystem cConnector does do.11:25
Hang on.
It didn't work perfectly, in the sense of not every JAX-RS endpoint worked right, but it found objects and stored them.
* VincentNG leaves11:40
* kaarefc joins11:41
<pivotal-bot____>Chris Beer edited "dsid/fixity stack trace when clustered" https://www.pivotaltracker.com/story/show/4767533711:50
Chris Beer edited "dsid/fixity stack trace when clustered" https://www.pivotaltracker.com/story/show/47675337
Chris Beer added "Every node should not fire EventBus events, etc, etc when a change is made on a different node." https://www.pivotaltracker.com/story/show/4773057511:51
Chris Beer edited "Every node should not fire EventBus events, etc, etc when a change is made on a different node." https://www.pivotaltracker.com/story/show/47730575
Chris Beer added "Update LowLevelStorageService to be cluster-aware" https://www.pivotaltracker.com/story/show/47730607
Chris Beer edited "Update LowLevelStorageService to be cluster-aware" https://www.pivotaltracker.com/story/show/47730607
Chris Beer added "Expose cluster health via REST API" https://www.pivotaltracker.com/story/show/4773068911:52
Chris Beer edited "Expose cluster health via REST API" https://www.pivotaltracker.com/story/show/47730689
Chris Beer added "Expose cluster numCopies as a java property" https://www.pivotaltracker.com/story/show/4773083311:54
Chris Beer started "Expose cluster numCopies as a java property" https://www.pivotaltracker.com/story/show/47730833
Chris Beer finished "Expose cluster numCopies as a java property" https://www.pivotaltracker.com/story/show/47730833
Chris Beer added comment: "Looking good. Restarted futures2 and it rejoined the cluster just fine.11:55
INFO 16:47:36.842 (JGroupsTransport)..." https://www.pivotaltracker.com/story/show/45565303
Chris Beer added "fcrepo 4.x needs some high-level cluster management logic to prevent incompatible versions of fcrepo from running in t..." https://www.pivotaltracker.com/story/show/4773103511:56
Chris Beer edited "fcrepo 4.x needs some high-level cluster management logic to prevent incompatible versions of fcrepo from running in the same cluster" https://www.pivotaltracker.com/story/show/47731035
Chris Beer edited "Upgrade to ISPN 5.3 (when available) to fix clustering work-around." https://www.pivotaltracker.com/story/show/4763933511:57
Chris Beer added comment: "Updated https://wiki.duraspace.org/display/FF/ModeShape+Clustering with current information" https://www.pivotaltracker.com/story/show/4440465312:05
Chris Beer added "Figure out if we can kill jgroups-fcrepo-tcp.xml (because it is suspiciously similar to the ISPN-distributed jgroups-t..." https://www.pivotaltracker.com/story/show/4773187112:06
Chris Beer edited "Figure out if we can kill jgroups-fcrepo-tcp.xml (because it is suspiciously similar to the ISPN-distributed jgroups-tcp.xml)" https://www.pivotaltracker.com/story/show/47731871
Chris Beer added "Test <UDP> and <MPING> on the futures cluster." https://www.pivotaltracker.com/story/show/4773191512:07
Chris Beer edited "Test <UDP> and <MPING> on the futures cluster." https://www.pivotaltracker.com/story/show/47731915
* nbanks leaves12:20
<ajs6f>cbeer or anyone who knows about Nexus: I'm getting:12:23
[ERROR] The project org.fcrepo:fcrepo-bagit-modeshape-federation-connector:4.0-SNAPSHOT (/Users/ajs6f/Documents/fcrepo4/fcrepo-bagit-modeshape-federation-connector/pom.xml) has 1 error
[ERROR] Non-resolvable import POM: Could not find artifact org.modeshape.bom:modeshape-bom-embedded:pom:3.2-SNAPSHOT in sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots) @ org.fcrepo:fcrepo:4.0-SNAPSHOT, /Users/ajs6f/.m2/repository/org/fcrepo/fcrepo/4.0-SNAPSHOT/fcrepo-4.0-SNAPSHOT.pom, line 62, column 19 -> [Help 2]
which looks like our MODE snapshot disappeared from Nexus.
Do we have to keep deploying it to keep it there, so to speak?12:24
<cbeer>not sure this is relevant, but our MODE snapshot isn't on sonatype12:25
it's on our own nexus server
<ajs6f>Is that Nexus server in the fcrepo4 POM?
<ajs6f>Hm. Weird. Not in my local copy. Ill try and pull that.12:26
Okay that worked. How odd. Oh, well.12:30
barmintor: watcing the i-test fail, are you seeing:12:31
TRACE 12:29:54.547 (ObjectService) Executing getObject() with pid: BagItFed1
TRACE 12:29:54.548 (ObjectService) Executing getObjectNode() with pid: BagItFed1
TRACE 12:29:54.548 (PathService) Executing getObjectJcrNodePath() with pid: BagItFed1
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 6.278 sec <<< FAILURE!
* nbanks joins12:36
* nbanks leaves12:41
<barmintor>ajs6f: yes12:45
from the Objectservice12:46
<ajs6f>Here's what's weird— FedoraObjects.getObject is where that starts, right?
But that calls:
which _doesn't_ call getObjectNode()!?12:47
So why is getObjectNode being called at all?
Ive been staring at this since I put the log up here.
I can't see it.
<barmintor>it's getting called by the http-api12:48
<barmintor>via the FedoraObject
stupid ajs6f.
I just wasted a lot of time.
<barmintor>well, stop it. you've got work to do.12:49
<ajs6f>Yeah, I've got to make this i-test pass. And I'm no closer.
Sorry— sys-test.
Too much of this is going on in MODE.12:50
I'm on the verge of going to full-on debugger assault.
<eddies>sigh. i can't get cargo to deploy a non-war artifact
doesn't seem like it's supported12:51
<ajs6f>Did you get any word from the Cargo people?
<eddies>no. but the docs don't indicate any support for it
<ajs6f>Maybe ask. Then file a ticket and walk away.
"A man's got to know his limitations."
<eddies>the only support cargo has for an exploded war involves cargo itself as responsible for creating the exploded directory from the war artifact: http://cargo.codehaus.org/Static+deployment+of+expanded+WAR12:53
corroborated here: http://cargo.996258.n3.nabble.com/Exploded-WAR-deployment-td1266.html
<ajs6f>But we don't have a WAR artifact, right?
<eddies>the fact i think of dirty hacks like creating a dummy WAR submodule to reference worries me12:54
ajs6f: no, fcrepo-http-api and generator-dc, generator-rdf are not wars
<ajs6f>Well, ^%**^% 'em. &^%&(^ 'em in the ear.12:55
<eddies>i'm going to call it a dead-end. it's honestly a bit weird anyway what's going on to support the jetty-plugin anyway12:56
<ajs6f>So the essential goal here was to do sys-tests w/ those modules?
<eddies>we have a src/test/resources/WEB-INF/web.xml in each module that has integration tests
and i stepping back, at least for the time being where our CDI future is uncertain, i think having spring manage the process is the right choice12:57
Hypothetically, if our modularization-fu is really powerful, the only place this should matter is in the API modules.
<eddies>i suppose for the fcrepo-webapp module, we could still explore cargo
<ajs6f>i-tests in other modules _shouldn't require a web container_.
<ajs6f>So our fu is not as fufull as is wanted.
Isn't this the habit of writing i-tests as sys-tests?12:59
(Coming back to bite us?)
<eddies>fasseg made the observation that in general, he's in favor of i-tests actually being in a separate maven module altogether
<ajs6f>Yeah, but that scares me.
<eddies>i'm a bit emotionally scarred by how we did that in fcrepo3
<ajs6f>That _encourages_ writing i-tests as sys-tests.
<eddies>that too
<ajs6f>If you keep 'em in the module concerned, you are forced to recognize when you cross module bondaries.13:00
That doesn't mean you do anything about it.
But it's like rubbing a puppy's node in its own mess.
It's a learning experience.
<eddies>in any case, i think that isn't the pattern we want: as we move some of those modules out of fcrepo4 (e.g. generator-dc/rdf) you're still going to want i-tests and i definitely don't want to set a pattern for developers to make multi-module projects just to write i-tests13:01
I had to do that for experimental fcrepo3 modules.
I made me even more irascible than I normally am.
<eddies>but from a Getting-Shit-Done perspective, I see the immediate attractiveness of having a single i-test module—since we could then just ignore this lifecycle business13:02
<ajs6f>Yyyyyyeaaaahhhh….. hm.
I mean,
<eddies>but i'm going to hold out hope that we can springify the i-tests and be done
<ajs6f>the right thing to do is to rewrite all of the itests in the modules.
So that none of them require HTTP.
(Except the HTTP API , of course.)13:03
<eddies>didn't barmintor jump through all sorts of hoops just to get the i-tests to work with http?
<eddies>or maybe that was just in the http-api module
i dunno. i remember him whinging about something like that last sprint ;-)13:04
<ajs6f>I'm starting to feel like part of what this is about is the lack of a tru module managment framework.
<eddies>i thought you were going to osgify everything for us, adam?
<ajs6f>Still waitiing to decide what "everything" is!
There seem to be a couple of themes here: modulariation, testing boundaries, kernelization.13:06
I'm going to put some of this on the agenda for the face-to-face, okay? We can take it off or alter it as we get closer.
<ajs6f>I somehow feel like this isn't going to be perfectly resovled before we get there. :)
<eddies>hey, you're coming by land to salem, right?
<eddies>so you can bring apple brandy no problem, eh? =)
<ajs6f>Already on my list.
<ajs6f>It's a good breakfast brandy.
<eddies>heh. reminds me i need to sort out how we're going to do coffee there
ok. i have call with the property owner in a few mins. bbl
<ajs6f>How _you're_ going to do coffee. Some of us are tea-drinkers
<eddies>i have three very nice teas i bought last week in china13:09
<ajs6f>What are they?
<eddies>a "long leaf" green tea, a white tea and a pu-er tea
<ajs6f>Ooh, I haven't had a good musty well-aged puerh in a long time.13:10
<eddies>i'll bring some. those are actually the easiest to carry
<ajs6f>Yeah, light and dry.
I hate carrying liquids.
<eddies>wouldn't hurt to remind me again as the date approaches =)
ok. afk
<ajs6f>You remind me about the brandy.
good luck.
<escowles>can anyone tell me the Right Way to configure a mixin in a separate module?13:18
i've got kitchen-sink checked out and if i put an updated repository.json in the right place, it will invoke the sequencer, but it's not finding the new mixin properties13:19
<ajs6f>You'll want to put it in the CND file that mode looks at to configure its node types.13:21
afk, but back in five.
* VincentNG joins13:23
* ap2972 joins13:24
* ajs6f leaves13:26
* ajs6f joins13:28
So that isn't done in kitchen-sink, but in fcrepo-webapp, which kitchen-sink overlays (we got to do something less confusing than that)13:30
in src/main/resources/config/fedora-node-types.cnd13:31
Although I think you could override that in kitchen-sink (not sure, I haven't really done much with kitchen-sink).
Are you trying to demonstrate functionlaity? If so I would start by showing in webapp, which is less complicated and more explicit.13:32
fcrepo-webapp, that is.
<escowles>ok, and should i also update src/main/resources/config/repository.json to invoke the sequencer?
<escowles>that makes sense -- i thought kitchen-sink was the place to try out new external modules
<ajs6f>It's more the place to make them visible.13:33
I think.
Again, this is really about the fact that our modularization framework comprises… Maven builds.
-1 to us.
<escowles>i have to say the maven build system is a little like pinball -- put something somewhere, let stuff bounce around for a while, and maybe something good happens13:34
<ajs6f>Maven works really well, when you use it for what it was built for— builds.
Are you just trying to write sys-tests or i-tests?
<ajs6f>Okay, at least in theory, you don't need to go outside your own module for that.
And in truth, that's what I've been able to do.
Have you looked at how i-tests in other modules are set up?13:36
You don't want to package up your mmodules
with a bunch of others to test them.
You want to use a testing framework (for now, Spring test).
And let maven just draw in the dependencies you need.
<escowles>i looked at the glacier sequencer module
<ajs6f>Take a look at how the i-tests/sys-tests in fcrepo-http-api work.13:37
Hm. I don't know that one.
Let me go look at that.
Is that:
but at least in theory, i should be able to pull the webapp and its dependencies into my module, and run the tests and stuff there with updated config, without having to update fcrepo-webapp, etc., right?13:38
<ajs6f>Hm. I'm not sure that's true.13:39
In fact, I don't think it is.
I think you would have to overlay fcrepo-webapp with a dependency to your module to do that.
<escowles>ok, i thought that was the point of kitchen-sink, but i'm really just grasping at straws here
<ajs6f>It looks like barmintor hooked into the MODE i-test apparatus for the Glacier stuff, so I'm going to let him comment on that.
I've never done that.13:40
To my understanding, and I welcome correction from cbeer/eddies on this, kitchen-sink is just meant to deliver everything all at once to show it off live for the demo sites.
It's not meant to be part of any dev path.
<escowles>ok, then kitchen-sink is not what i should be using for initial development -- but i still don't really know how i develop code in a separate repo/module and then get fcrepo-webapp to use it
<ajs6f>If you are not aas far as having passing i-tests or sys-tests, don't worry about fcrepo-webapp. Not your problem.13:42
Just worry about having in-module i-tests.
And we can circle around again when it's time to integrate your work into something larger.
Does that make sense?13:43
<escowles>ok, so taking a step back (from somone who's never done maven or TDD in java) -- don't i need the webapp, modeshape, etc. running to get all the infrastructure that the sequencer relies on?
Well, you need MODE.
And you may need other fcrepo modules,
if you are doing sys-tests.
But you don't (or shouldn't) need fcrepo-webapp.
For example,
look at fcrepo-rss or fcrepo-jms or fcrepo-http-api.13:44
None of them know anything about fcrepo-webapp.
They assemble the stuff they need for i-tests/sys-tests via Maven dependencies and the Spring test framework.
<escowles>ok, this looks very different from the other I-tests i was looking at13:45
<ajs6f>Ideally (and to some extent, in reality now) i-tests occur between modules, and sys-tests occur between.. well, lots of modules, but fcrepo-webapp is an artifact that just gathers up a lot of modules into a deployable artifact.13:46
Yeah, I'm not sure what's going on in the Glacier stuff, but it _is_ two months old. Virtually from the time of the Pyramids. :)
<escowles>it seemed like the most obviously-similar code, but that doesn't make it the best example to follow13:47
<ajs6f>Fair enough.
* nbanks joins
<ajs6f>Again, I'll let barmintor comment on where that stuff actually is.
When he gets back.
But the fcrepo-htt-api stuff is a good example.
BTW… have you seen Spring before?13:49
Cause… there's a good bit of it.
<escowles>i haven't done any development with it - but one of our projects uses it so i've seen it a little13:50
<ajs6f>Okay, cool.
afk back in a bit13:57
* ajs6f leaves
* ap2972 leaves14:00
* ajs6f joins14:01
<eddies>ajs6f/escowles: kitchen-sink indeed is just for demo purposes14:04
although it is what we deploy on futures6, so the build we run hydra and islandora against too14:05
<ajs6f>eddies: right. the real question escowles is asking, I believe, was "How on earth do I do i-tests and sys-tests?"
(In a new extension module.)
<eddies>famous last words: you *should* be able to follow the example of fcrepo-http-api and the like14:06
<ajs6f>Check the logs— that's exactly what I said.
<eddies>there you go
<ajs6f>It _must_ be true!
* cbeer leaves14:50
* pivotal-bot____ leaves
* ajs6f leaves15:00
* bljenkins leaves15:27
* cbeer joins15:28
* bljenkins joins15:29
* ajs6f joins15:33
* pivotal-bot joins15:34
* ajs6f leaves15:41
* VincentNG leaves15:42
* ajs6f joins15:45
<escowles>ajs6f: i now have a better (and passing) unit test and an i-test based on the fcrepo-http-api i-tests. the i-test compiles, but maven doesn't seem to be running it15:48
<ajs6f>How re you onvoking maven
Also, do you have the failsafe plugin decalred in your maven pom?15:49
<escowles>i tried mvn test, integration-test, etc.
<ajs6f>You want mvn verify.15:50
_If_ you have decalred the failsafe pluging
<escowles>i don't have failsafe -- i'll copy that over from fcrepo-http-api
<escowles>excellent -- i now have failing i-tests -- thanks!15:51
<ajs6f>+1 to failing tests!
* ajs6f1 joins16:22
* ajs6f leaves
* kaarefc leaves16:37
* escowles leaves
* github-ff joins16:51
[fcrepo4] cbeer pushed 1 new commit to master: http://git.io/XJg3lQ
fcrepo4/master c9942b7 Chris Beer: explicitly disable lock striping
* github-ff leaves
<pivotal-bot>Chris Beer added "org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] " https://www.pivotaltracker.com/story/show/4775830716:52
Chris Beer started "org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] " https://www.pivotaltracker.com/story/show/47758307
Chris Beer added comment: "Trying: <locking useLockStriping="false" />" https://www.pivotaltracker.com/story/show/47758307
<ajs6f1> cbeer et al: is the repository size calculation now completely dynamic? IOW, can we get rid of any of that old code that kept a running counter in the repo?16:54
<cbeer>i believe that's right
<nbanks>I don't know but when I add objects I get a negitive repository size back?16:55
<nbanks>DEBUG 13:51:59.163 (RepositoryService) Previous repository size: -38845
DEBUG 13:51:59.180 (RepositoryService) Current repository size: -43088
something like that
I'm a couple of commits back though16:56
<ajs6f1>Well, if we're confident we _can_ do a dynamic technique, we should still get rid of the old code and narrow down the problem.
Can you rebase and check again?
<nbanks>sure one sec
<pivotal-bot>A. "Erdocamic" Soroka added "Remove old code related to keeping track of repo size" https://www.pivotaltracker.com/story/show/47758917
<nbanks>"DEBUG 21:27:05.434 (FedoraObjects) Attempting to ingest with pid: test:117:00
DEBUG 21:27:05.434 (SessionFactory) No authenticated user found!
DEBUG 21:27:05.549 (SimpleObserver) Putting event: Node added at /objects/test:1 by <anonymous> on the bus.
DEBUG 21:27:05.564 (JMSTopicPublisher) Received an event from the internal bus.
DEBUG 21:27:05.574 (FedoraObjects) Finished ingest with pid: test:1
DEBUG 21:27:05.889 (LegacyMethodEventFactory) Constructed serialized Atom message from event.
DEBUG 21:27:05.890 (JMSTopicPublisher) Transformed the event to a JMS message.
DEBUG 21:27:05.891 (JMSTopicPublisher) Put event:
ID:dev.local-41654-1365535555555-3:1:1:1:1 onto JMS.
Didn't print out this time?
<ajs6f1>I think the dynamic calculation wouldn't.
So you're now using it, perhaps. Try getting a repo description. Is it sane?
<nbanks>seems reasonable has 0 as the size for an empty object I posted, one sec I'll test it with actual data17:01
<ajs6f1>Actually, we've had this discussion before: an empty object still has a footprint (basic properties, etc.) Some have said that it still ought to be measured. Others have said that since the properties tend to be very small compared to bitstreams, it doesn't mater...17:02
<nbanks>I posted a 1MB file to a datastream seemed to work, but the size states it is 87 something on the description page.17:05
<bljenkins>Project fcrepo-kitchen-sink build #153: SUCCESS in 3 min 17 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-kitchen-sink/153/
<ajs6f1>Hm. Weird.17:06
Well, I've got to run, but I'lll follow up on this tomorrow.
See y'all later.
* ajs6f1 leaves
<bljenkins>Project fcrepo-fixity build #145: SUCCESS in 3 min 45 sec: http://ci.projectblacklight.org/jenkins/job/fcrepo-fixity/145/
<nbanks>dsSize is 498 oddly enough
* travis-ci joins17:11
[travis-ci] futures/fcrepo4#344 (master - c9942b7 : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/7ac900c2d1a2...c9942b7ab3c1
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/6197885
* travis-ci leaves
* ap2972 joins17:50
* nbanks leaves19:17
* github-ff joins19:54
[fcrepo-bagit-modeshape-federation-connector] barmintor pushed 1 new commit to master: http://git.io/eBtZbg
fcrepo-bagit-modeshape-federation-connector/master 335379c Benjamin Armintor: rolling back to just a single bag, almost working
* github-ff leaves
* ap2972 leaves21:32
* nbanks joins23:25
* nbanks leaves23:29

Generated by Sualtam