Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
<pivotal-bot>Edwin Shin accepted "Create BagIt bag-info.txt -based ExtraPropertiesStore" https://www.pivotaltracker.com/story/show/4738331500:55
Edwin Shin accepted "Create sequencer to test large files" https://www.pivotaltracker.com/story/show/47374193
Edwin Shin accepted "Add fcrepo-sequencer-archives to kitchen-sink" https://www.pivotaltracker.com/story/show/4783028100:57
Edwin Shin added comment: "See: https://github.com/ModeShape/modeshape/blob/master/modeshape-jcr/src/test/java/org/modeshape/jcr/Abstrac..." https://www.pivotaltracker.com/story/show/4614232901:06
* eddies joins
* eddies leaves
* eddies joins
* kaarefc joins01:20
* kaarefc leaves01:37
* kaarefc joins
* kaarefc leaves01:58
* kaarefc joins02:28
* kaarefc leaves02:47
* kaarefc joins03:01
* kaarefc leaves03:06
* kaarefc joins
* kaarefc leaves04:17
* eddies leaves04:30
* eddies joins04:47
* eddies leaves
* eddies joins
<fasseg>oh my box was running the whole weekend...06:06
morning guys
wry if I didn't answer ;)06:07
eddies: So i tried jersey in jboss as integration testing platform, which brings CDI, and of course it works well....06:11
i tried the jax-rs 2.0 beta too that resulted in the same ussues
<eddies>uh huh
<fasseg>RestExpress -> i had two beans in the lifycycle, i have no idea why :/06:12
err i was talking about jersey 2.0 beta earlier
the jax-rs 2.0 impl
and a test run on oracle app server
went well too with jersey
switching the @inject to @autowired worked well too btw, and was not much work...
and can be undone quickly i guess
quick talk maybe, where to go from here?06:15
<eddies>i don't have my headset with me—working at a cafe
<fasseg>ah ok...06:16
<eddies>let me make sure i'm following:
what works is:
<fasseg>I can push my workday in the late evening...got a lot of sunshine, if you wanna talk later...
<eddies>i) replace CDI @Inject with Spring's @autowired06:17
ii) jersey (version?)
jersey-spring 1.0 and jersey-servlet 1.17.106:19
so jersey 1.17.1
<eddies>and a) how do the ITs run for non-war modules (SpringJunit4Runner?) b) for war modules (e.g. fcrepo-webapp?
<fasseg>i created ars by hand :/
for the jboss and glassfish run
<eddies>for all the non-war modules? generator-dc, generator-rdf, http-api, etc?06:20
<fasseg>TJWS runs too..
so integration tests without wars run with TJWS
and @Autowired
<eddies>actually, i just thought of something...06:21
<fasseg>but for glassfish/jboss pre integration post integration start/stop i created wars by hand for tests
<eddies>i think you weren't on scrum the other day when i mentioned this
<fasseg>right i mas missing friday
<eddies>oh wait. nevermind. different problem, doesn't apply here06:22
ok, so we don't need to create wars if we're not running our ITs against glassfish/jboss06:23
<fasseg>right but with glassfish/jboss you get to use @inject
without i had to switch to spring annotations06:24
<eddies>i think i can give up @inject for @autowired until we make a complete move to CDI
although you should raise this on ff-tech in case others have feedback
<fasseg>i agree, like i said the switch is done quite fast
<eddies>but seems to me that we're only half-assing cdi right now anyway, so no big loss06:25
<fasseg>okay i'll post it..
and I also feel we should run some integrtion tests against tomcat since it seems to be the most commonly used platform, at least in my experience
<eddies>i'd rather lose @inject than artificially create wars
<fasseg>yes, and curating the assembly plugin can be a nightmare06:26
<eddies>so what about tjws vs the springjunit4runner for the non-war modules?06:27
what are issues favoring one over the other?
<fasseg>sems to work for a test project for me
I can probably test this in about 1-2 hours on fcrepo
well you know 3-4h ;)06:28
developer hours...
but theoretically it means only switching the annotations and adding the TJWS bean to the SpringJunit4Runnner config06:29
but I did not yet run a fcrepo4 rest service on top of TJWJ jaxrs server, but it works with an @autowired service which generated a helloworld response...06:30
and it seems TJWS is mem only and you can just kill it's vm to shut it down ;)06:31
so there is no issues of starting multiple TJWS instances back to back06:32
<eddies>ah, so w/ tjws, you're still using the springjunit4runner?
I've written a simple bean wrapper for the TJWS server06:33
so you can configure a <bean class="wrapper" start.method="start" stop-method="stop><..port config etc..></bean>
let me gist it06:34
<eddies>wait. this is all w/ jersey or still resteasy?
tjws is for use w/ resteasy, no?
i thought w/ jersey, grizzly was the preferred test container
<fasseg>yes, but i had the same @Inject problems with grizzly06:36
so jersey would have no references to our injected services06:37
wrapper bean config: https://gist.github.com/fasseg/5a32b3a4b144af6f3a5406:38
and the wrapper class: https://gist.github.com/fasseg/5387237
<eddies>ok. and does component-scanning still have to be disabled?06:41
<eddies>we can have component-scanning? =)
<fasseg>you can disbale resteasy's internal scanning in the web.xml
and use <context:component-scanning base-package="..." />
from spring
<eddies>ok. that sounds like a pretty good outcome from what i'm understanding06:44
<fasseg>i have only this in my rest config: <context:component-scan base-package="net.objecthunter.doodle.resources, net.objecthunter.doodle.services" />
<eddies>that's great
<fasseg><- humbled
<eddies>ok, so are yu good to proceed for now then?
<fasseg>I think so: I'll try to get all the test to run using spring annotations and TJWS06:45
then throw it all away and rebase on top ;)
I think my branch is 13 days old now :/06:46
<fasseg>that's gonna be quite the merge otherwise
okay then, have fun and enjoy the café06:47
<eddies>here's my possible coffee list for next week, a shame you won't be there: https://wiki.duraspace.org/pages/viewpage.action?pageId=3464747006:49
k. talk to you later
<fasseg>heh I've never even heard of those... but yeah it would have been nice, but I'm thinking of going on the trip in private next time....But don't tell Matthias ;)06:51
* eddies leaves06:54
* eddies joins07:02
* eddies leaves
* eddies joins
* VincentNG joins08:01
<pivotal-bot>Esme Cowles estimated "Measure time to ingest files of varying sizes & run a sequencer comparing the REST API vs BagItFileSystemConnecto..." https://www.pivotaltracker.com/story/show/4737603708:31
Esme Cowles started "Measure time to ingest files of varying sizes & run a sequencer comparing the REST API vs BagItFileSystemConnector" https://www.pivotaltracker.com/story/show/47376037
* escowles joins
* kaarefc joins08:36
* nbanks joins08:58
* kaarefc leaves09:06
* kaarefc joins09:34
<pivotal-bot>Chris Beer added comment: "Yeah, that's incredibly annoying. I wonder if there's a java property we can set somewhere to control those....." https://www.pivotaltracker.com/story/show/4614232910:11
<cbeer>eddies: now it's our turn to do interviews all week. probably best if someone else runs standup10:14
* kaarefc leaves10:18
* ajs6f joins10:23
<pivotal-bot>A. "Vebelliant" Soroka edited "Create ModeShape BagIt federation connector" https://www.pivotaltracker.com/story/show/4708984310:40
A. "Vebelliant" Soroka edited "Create a design document for the BagIt connector describing our implementation, requirements, and limitat..." https://www.pivotaltracker.com/story/show/47646459
A. "Vebelliant" Soroka edited "Tomcat 7 on futures6 Permgen Exception on re-deploy" https://www.pivotaltracker.com/story/show/46543301
<eddies>cbeer: that's fine. can you make standup today, however? i want everyone on since it's day before sprint-end10:46
<cbeer>eddies: nope. but i have nothing new to report either. working on a modeshape forum post to figure out who to blame for the deadlock problem.10:51
* ajs6f leaves10:53
* ajs6f joins10:57
<eddies>barmintor: just waiting for you11:00
i lied. escowles too11:01
<barmintor>I'm there
<VincentNG>i'm in a collaboration room with 4 other people so it might get load when i talk....sorry ahead of time :-)11:02
* github-ff joins11:10
[fcrepo-authn] ajs6f created master (+1 new commit): http://git.io/4E9gLg
fcrepo-authn/master e608fd6 ajs6f: Initial commit holding place for authN architecture
* github-ff leaves
<pivotal-bot>A. "Vebelliant" Soroka unstarted "Move SessionFactory out of fcrepo-http-commons into a new fcrepo-authn module with default impl for HTTP." https://www.pivotaltracker.com/story/show/4789590111:12
A. "Vebelliant" Soroka edited "Move SessionFactory out of fcrepo-http-commons into a new fcrepo-authn module with default impl for HTTP." https://www.pivotaltracker.com/story/show/47895901
<barmintor>I'm not complaining about the priorities, but I hate feeling pushed around by our jaxrs impl11:20
barmintor: did you see the new apache jaxrs impl? maybe we should try that!?!?
<ajs6f>Apche Wink?
<ajs6f>It does look quite clean and nice, but I'm not sure how it stands WRT to the issues of DI and context that have been dogging us.11:24
<barmintor>You remember in Pulp Fiction, when Samuel Jackson is yelling "Motherf***er, say "What" one more time!"
<ajs6f>Say Wink.
<barmintor>That's kind of how I feel about JAXRS impl right now
<cbeer>fcrepo 4: bring your own rest endpoint
<ajs6f>Do you have a wallet named Bad Motherf**)?
<barmintor>ajs6f: :P11:25
<ajs6f>As always, your taste is impeccable.
<barmintor>ajs6f: don't lose the thread on your projection comment11:27
<ajs6f>Yeah, we can circle back around that question.
barmintor: don't forget to remember!11:48
<barmintor>the size/checksum properties as the ds node11:51
ajs6f eddies ^^
<ajs6f>on the ds node
<cbeer>what's the question?
<barmintor>MODE's properites are on the jcr:content node11:52
purs are on the DS node
<barmintor>eddies: I need to finish these tests before I can pivot11:56
<pivotal-bot>Edwin Shin edited "Watcher for federated filesystem that triggers events" https://www.pivotaltracker.com/story/show/4738061912:01
<eddies>?barmintor: were we supposed to remind you of something12:08
<barmintor>TOTAL FAILURE
<barmintor>It was in regard to the required properties
ajs6f asked why the node wrappers weren't enforcing it on ff-tech
the answer is: they are, when they can (ie, the object node is writeable)
but in federations, you don't get nodes
you get DocumentWriters
<ajs6f>barmintor: are connector.changeManifest() and connector.changeTagFile() the methods that get called as callbacks when a file changes on the filesystem?12:10
<barmintor>and the node wrappers don't know about those
ajs6f: yes
<barmintor>ajs6f: see also that ManifestMonitor class
<ajs6f>Yeah, that's where I found them/
Now I'm gong to replace:12:11
// do some manifest stuff
<eddies>so do we some sort of nodewrapper factory to deal w/ nodes vs documentwriters?
<ajs6f>with some manifeest stuff.
<eddies>and perhaps more importantly, can we punt on that for now?
<barmintor>eddies: I thnik we have to. I just want to make sure we know where the issue is coming from12:12
<ajs6f>crap. a federation connector doesn't actually have access to the repository objects. Have to figure out how to trigger events indirectly, or find some way to get the repo injected in.12:13
<barmintor>yep, same issue
<pivotal-bot>Edwin Shin added "Nodewrapper factory for nodes & documentwriters" https://www.pivotaltracker.com/story/show/4809111912:15
<eddies>barmintor: i'm just making you the owner for now on ^^ so you remember to clarify/correct the description of the work later12:16
<pivotal-bot>Edwin Shin delivered "Create clustered fcrepo4 repository on futures2-5" https://www.pivotaltracker.com/story/show/4764846112:24
Edwin Shin accepted "Create clustered fcrepo4 repository on futures2-5" https://www.pivotaltracker.com/story/show/47648461
<eddies>cbeer: where is the work for https://www.pivotaltracker.com/story/show/47730833? (i.e. so i can review it to be delivered/approved)12:26
<pivotal-bot>feature: Expose cluster numCopies as a java property (finished) / owner: Chris Beer
<pivotal-bot>Edwin Shin added comment: "see: https://github.com/futures/fcrepo4/blob/master/fcrepo-webapp/src/main/resources/config/infinispan_config..." https://www.pivotaltracker.com/story/show/4773083312:27
Edwin Shin accepted "Expose cluster numCopies as a java property" https://www.pivotaltracker.com/story/show/4773083312:28
<ajs6f>Urb. Sequencers _don't_ listen to JCR Events. They listen to internal MODE events.12:42
See org.modeshape.jcr.Sequencers implements ChangeSetListener12:43
<eddies>ajs6f: i think we had discussed that early on in the sprint12:49
<barmintor>I sure do like trying to test all these damned private members set by reflection, MODE!12:50
NPEs everywhere, YAY
<ajs6f>eddies: No, we had discussed how to trigger JCR Events by artifically creating ChangeSets. That won't do us any good, here, tho.12:54
<barmintor>I think I'm learning the lesson that JSON is just a shitty way to configure Java objects13:12
<ajs6f>barmintor: I'm starting to wonder how MODE utests its own stuff.
<barmintor>ajs6f: I don't think they do it very consistently
<barmintor>they also probably inherit some baggage from ISPN13:13
and from their own previous notions of config
<ajs6f>We might want to look and see if ISPN offers anything there.
<barmintor>do they!
<ajs6f>o yeah, I knwo about that— I meant if they offer anything to help w/ mocking.13:14
<barmintor>I poked a round a little, but I think it's really structural
<ajs6f>Its not just encapsulation— it's sufforcation.
<barmintor>yeah- when the way your objects will be init'd is this json -> reflection -> private member route13:15
it puts a huge constraint on how you can implement13:16
It's not even just a testing concern
* ajs6f leaves13:19
* ajs6f joins13:20
Best Constructor Signature Ever:
protected ExecutionContext( SecurityContext securityContext,
NamespaceRegistry namespaceRegistry,
PropertyFactory propertyFactory,
ThreadPoolFactory threadPoolFactory,
BinaryStore binaryStore,
Map<String, String> data,
String processId,
TextDecoder decoder,
TextEncoder encoder,
StringFactory stringFactory,
BinaryFactory binaryFactory,
ValueFactory<Boolean> booleanFactory,
DateTimeFactory dateFactory,
ValueFactory<BigDecimal> decimalFactory,
ValueFactory<Double> doubleFactory,
<ajs6f> Argggg. All of the ways to get at the ChangeBus of the repository (the place to trigger Changes) are "protected".13:35
The only obvious way around is to trivially subclass JcrRepository and expose those members.
Or to do something icky with reflection.13:36
<barmintor>you could have your calls to those methods go through a proxy we write and assign to the relevant pkg
that's what I usually do
<ajs6f>Hm. Have we found a use case fo Java's dynamic proxy?13:37
<barmintor>Maybe. I usually just have static methods in an accessor class.
* barmintor shrugs
<ajs6f>A proxy that would expose javax.jcr.Repository plus a method or two to hook the ChangeBus.
<barmintor>but yeah, you could also do a subclass that does nothing but widen the permissions on the methods13:38
<ajs6f>That's what I was first suggesting.
But wait…. crud.
<barmintor>but if the member is private on the superclass, you get into
<ajs6f>It's protected.13:39
<barmintor>a weird name-shadowing space.
<ajs6f>So that's okay.
But how does our connector _get_ a proxied repo.
MODE controls the lifecyle of this component.
<ajs6f>Doing that funky JSON-reflected-into-private-fields config thing.
It's not a bean.13:40
<barmintor>you keep your proxy locally, inject the repo, and wrap it on init
but be careful to clean it up on shutdown
<ajs6f>So it can't be a simple subclass.
<barmintor>or you'll leak memory
<ajs6f>And I can't inject it with Spring.
I'll have to get it from a fresh ModeShapeEngine.
<barmintor>We should just move our classes into modeshape packages
<barmintor>everything except the API
I'm kind of serious
<ajs6f>I understand.
But we should, no matter what, have some kind of discussion with them about it.
<barmintor>I guess we should be telling MODE this stuff
err… yeah
<ajs6f>I understand that they want to define their public API.13:42
But they're not letting people like us help them figure out where to take it.
Well, I'll start by trying to see if I can get this proxy waltz going.13:43
I guess I'll just do it inside the connector code, for now.
<barmintor>ajs6f: https://gist.github.com/barmintor/7e40e21ff6654c159e0e13:47
<ajs6f>What is that the mocking behind?13:49
Oh, the connector.
barmintor: just a heads-up: if this works, we will need to thikn about when/how changes get emitted from the connector, because the change objects themselves get created like:14:04
public void nodeCreated( NodeKey key,
NodeKey parentKey,
Path path,
Map<Name, Property> properties )
so we'll need to know the path, the nodekey (?), the parent key (do we even know that) and any prperties. we may be able to get away without some of this, but I don't yet know14:05
and we'll have to make some kind of translation between the kind of events that the filesystem monitor gets from the Java file watching apparatus and these kinds of more JCR-specific events.14:08
barmintor: is the connector IT test supposed to be passing now?14:12
<barmintor>it was last time I checked it in14:13
via mvn verify
* ajs6f leaves14:14
* nbanks leaves14:18
* ajs6f joins14:19
Ookay. Hmm.14:23
Let me try that again.14:24
Yeah, okay. I must have done _something_ to break it. Anyway, no prob.
Ah, I see. The question now is how do I get a hold of the running repository.14:29
<barmintor>oh, because the repo is injectable, but connectors aren't really beans14:30
<ajs6f>Right. They're ModeShape components.14:31
<barmintor>Hey, what if we switched back to Lily?
Lily + RestEasy
<barmintor>can you get the repo from the ExecutionContext?14:32
<ajs6f>Just looking at that. I don't think so.
<ajs6f>I think you're not supposed to be able to.
Once again, we find ourselves fighting our platform.
we could do some jndi shenanigans
<ajs6f>Yeah, it's starting to look that way.
* barmintor ducks
<ajs6f>Which won't play well with our idea to put MODE's HTTP API alongside ours.
But oh well.14:34
<barmintor>is there a way to change the bootstrapper to use a subclass of Connectors?14:35
<ajs6f>_Their_ bootstrapper?14:36
maybe we can config it
<ajs6f>They don't expose their wiring at all, that I can see.
But maybe there is.
<barmintor>otherwise why would it have protected methods :D
<ajs6f>I don't understnad much about their views on vsiblity.14:37
<barmintor>me neither
I think we should actually see if we can get a response from them on the roadmap for this. and offer them a pull request for a change
<ajs6f>Meaning opening access from a Connector to the repo?14:38
<barmintor>at least, explain the issue and see if they have an opinion about it
<ajs6f>Yeah, I asked earlier today on the IRC, but no answer yet.14:40
Question for ModeShape folks: We're interested in creating interaction between federation and sequencing. Is it possible to set up a federation connector so that it could emit repository Changes when some resource in its remit undergoes change, thereby triggering sequencing?
<escowles>there's a modeshape bug for this (connector should be able to propagate changes in external systems via events: https://issues.jboss.org/browse/MODE-1710), but no action on it other than bumping it from each release14:44
<ajs6f>Nice catch, escowles.14:45
<barmintor>It's a blocker, huh?
<ajs6f>Everybody log in and vote.
<barmintor>For three releases?
<ajs6f>Lord knows that as soon as an fcrepo3 issue gets even one vote, we're on it like African bees.14:46
* nbanks joins14:47
<ajs6f>Oh, foo. can't do JNDI, because we don't have an initial context available. In fact, if we want to run in servlet containers, we aren't supposed to rely on one being present.14:49
barmintor: you're asking questions in #modeshape from now on.14:50
you get answers.
* nbanks leaves14:53
* nbanks joins15:04
<ajs6f>barmintor: Sounds like I'm going to fork and make a pull request...
* eddies1 joins15:08
* eddies1 leaves
* eddies1 joins
* eddies leaves
<VincentNG>barmintor: I'm seeing this exception when grabbing datastreams under the BagItFed1 object15:11
any idea?
<barmintor>There's an iterator impl that is being asked for a next element when there isn't one15:12
<VincentNG>i got the mock bagit store from your project: https://github.com/futures/fcrepo-bagit-modeshape-federation-connector/tree/master/src/test/resources/test-objects
next child element tho right? well, there should be children under your BagItFed1 node tho isn't there?15:14
<barmintor>one sec
Yes, there ought to be15:25
that for loop is buggy, though
it should be a while loop
anyway, yes
<ajs6f>Just use a goto.
<barmintor>ajs6f: hush15:26
VincentNG: where is this code being executed?
do you have a bagit connector included?
<VincentNG>yes, i have a bagit connector included15:27
i get it at ttp://localhost:8080/rest/objects/BagItFed1
so it's having trouble grabbing the datastreams (children)15:28
<barmintor>Well, my first instinct is to say that the connector is not working15:29
like, not being created or used
<VincentNG>which module does the connector need to be added to? http-api (cuz that's where it's being used) and maybe webapp right?15:30
<barmintor>ajs6f: https://github.com/futures/fcrepo-bagit-modeshape-federation-connector/issues/2
<ajs6f>barmintor: Nice catch.
<barmintor>ajs6f: now get to work :P15:31
<ajs6f>Put that whip down.
Damn. I make really good chopped liver. The secret is plenty of raw onion.15:40
<barmintor>I haven't had chooped liver in forever.
there's a place… what's it called
<ajs6f>You live in NYC.
How does this happen?
<barmintor>Sammy's Roumanian15:41
<ajs6f>There used to be Ratner's, on the lower east side. But it closed a few years ago.
dinosaur steaks, chopped liver made at the table
<ajs6f>Hm. I may have to go on a chpped liver tour of New York.15:42
To cure my anemia, of course.
<barmintor>"Not many restaurants keep a pitcher of chicken fat on every table as a condiment. "
<ajs6f>Oh, WOW.15:43
And my wife can have an egg cream.
<barmintor>that place is one of the few places that's both a tourist joint and just completely amazing15:45
ajs6f: the only problem is it's a little pricy: http://www.allmenus.com/ny/new-york/250096-sammys-roumanian-steakhouse/menu/15:48
<ajs6f>It sounds lovely. I think I'll go there with a wheelbarrow to roll home in.
<barmintor>heh, that's exactly how I felt when last I left the place15:49
<ajs6f>Meh. Who goes to NY as a tourist expecting to scrimp? It's like refusing to mix shmaltz into your matzah balls.15:50
It just doesn't taste as good.
<barmintor>well, that was useful, but it totally railroaded my afternoon15:52
<ajs6f>Sorry about that.15:53
I don't how you got such a response when I got crickets.
<barmintor>ajs6f: I'm about to push some unit tests and go to a meeting
<ajs6f>The gross Yankee soda? Not worth it.
<barmintor>I should be able to whittle a bit later this evening15:58
<ajs6f>Cool. I'm on to this new contract for eventing out of the Connectors.16:13
<cbeer>ajs6f: looks like you guys had a good conversation in #modeshape, huh?16:15
<ajs6f>barmintor did. I couldn't get arrested in the town without him.
<escowles>did everybody see the news about boston?16:17
<barmintor>news… about… boston
<ajs6f>Some kind of event at the Marathon?16:18
<escowles>barmintor: bombing at the marathon
<pivotal-bot>Vincent Nguyen added comment: "Initial work done here: https://github.com/vnguyen745/fcrepo4/tree/federated-content" https://www.pivotaltracker.com/story/show/4737377716:26
Vincent Nguyen added comment: "POST to /objects/pid/federate?from=FileSystem1" https://www.pivotaltracker.com/story/show/47373777
* VincentNG leaves16:28
<pivotal-bot>Vincent Nguyen added comment: "Couldn't get BagIt Connector working https://gist.github.com/vnguyen745/5390529 " https://www.pivotaltracker.com/story/show/4737377716:29
* github-ff joins16:30
[fcrepo-bagit-modeshape-federation-connector] barmintor pushed 1 new commit to master: http://git.io/fsvLQA
fcrepo-bagit-modeshape-federation-connector/master 61a2283 Benjamin Armintor: unit testing BagItConnector
* github-ff leaves
* ajs6f leaves16:35
* ajs6f joins
* escowles leaves16:43
<ajs6f>good night, all.17:13
* ajs6f leaves
* nbanks leaves18:31
<eddies1>barmintor: you there?19:20
* nbanks joins19:28
* nbanks leaves19:32
* eddies leaves19:57
* VincentNG joins22:45
* nbanks joins23:50
* nbanks leaves23:55
* nbanks joins00:01
* nbanks leaves00:05

Generated by Sualtam