Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* ksclarke leaves01:46
* fcrepo-bot joins08:14
* daines leaves08:42
* benpennell1 leaves08:59
* ajs6f1 joins09:07
* ermadmix joins09:11
* dwilcox joins
* tecoripa joins09:14
* tecoripa leaves
* gregjansen leaves09:22
* ksclarke joins09:41
* ajs6f1 leaves09:47
* dwilcox leaves09:48
* ajs6f1 joins09:49
* ajs6f joins09:57
* ajs6f1 leaves
<ajs6f>all: Just a reminder to add any committers' meeting agenda items here:10:07
Right now, we have a one-point agenda which could give us an awesomely short meeting.
* fcrepo-bot leaves10:09
* gregjansen joins10:10
* dwilcox joins10:11
* ajs6f leaves10:20
* ajs6f joins10:24
* daines1 joins10:25
* scossu joins10:30
* edInCo joins10:38
* fasseg joins10:45
* ajs6f leaves10:48
* ajs6f joins10:52
* jbaiter joins10:53
hi :-)
quick question: Is it possible to configure fedora so that some of the API-M methods do not require authentication?
we're trying to allow anonymous access on the "get*" methods10:54
<ajs6f>jbaiter: Yes, it is. You can do that directly in the Spring configuration for secutiry.
(By disabling some of the filters.)10:55
<jbaiter>hm, my colleague told me that he had already tried that, but some of the methods failed since Fedora tried to obtain the remote user for writing the audit
<ajs6f>I'm not sure I've ever heard of that. Can you send a message to the fedora-community@googlegroups.com mailing list?10:56
all: Committers' meeting on ReadyTalk.10:58
866-740-1260, 2257295
* ttappe joins
<jbaiter>ajs6f: will do once I have gotten more specific backtraces from the colleague or reproduced it myself, thanks :-)10:59
(he's here, btw, -> ttappe )11:00
<ajs6f>jbailter: Thanks, that would be great. Stacktraces are definitely in order here.
* nikhiltri joins11:02
<ajs6f>Sor far we have: A. Soroka, Stefano Cossu, Mike Durbin, Ed Fujikawa
and now Mike Daines11:03
* ksclarke just tuning in to lurk too11:04
<ajs6f>PLease do.
have mic off :-)
* benpennell joins
* mikeAtUVa joins
* martind_ joins11:06
* dwilcox leaves11:22
<jbaiter>ajs6f: I think we got it, our problems were based on a misunderstanding of the REST api .-)11:35
actually API-A does everything we want
we confused getDatastream with getDatastreamDissemination11:36
<ajs6f>jbaiter: Awesome. Glad you're back on the road!11:37
gregjansen: Let me know when you get the minutes into the wiki page, and I'll send out the usual email to the fedora-* lsits.11:41
* ttappe leaves11:43
<gregjansen>ajs6f: minutes are done. pls fill in Stefanno's first question, if you can remember it. Anybody else is also welcome to edit minutes11:53
<pivotal-bot____>A. "Torcrexilly" Soroka added "Improve event reporting facilities for node property changes" https://www.pivotaltracker.com/story/show/6660307211:58
A. "Torcrexilly" Soroka edited "Improve event reporting facilities for node property changes" https://www.pivotaltracker.com/story/show/6660307211:59
* nikhiltri leaves12:13
* ermadmix leaves12:15
<pivotal-bot____>Stefano Cossu added "Order RDF triples in REST API for property creation" https://www.pivotaltracker.com/story/show/6660500812:16
* ermadmix joins12:19
* ksclarke leaves12:38
* ksclarke joins
* ksclarke leaves12:39
* ksclarke joins12:41
* ermadmix leaves12:51
* edInCo leaves13:02
* ermadmix joins13:04
* ajs6f leaves13:05
* dwilcox joins13:28
* ermadmix leaves13:38
* scossu leaves13:45
<pivotal-bot____>Mike Daines edited "HTTPHeaderPrincipalFactory#getGroupPrincipals always returns the empty set" https://www.pivotaltracker.com/story/show/6645551013:46
Mike Daines edited "Improve documentation of PEPs" https://www.pivotaltracker.com/story/show/66460962
Mike Daines edited "Request attributes like IP address should be accessible to implementations of FedoraPolicyEnforcementPoint" https://www.pivotaltracker.com/story/show/66375794
* edInCo joins13:50
* dwilcox leaves13:53
* scossu joins13:56
* dwilcox joins14:04
* dwilcox leaves14:10
* dwilcox_ joins14:13
* dwilcox_ leaves14:23
* github-ff joins14:30
[fcrepo4] bbpennel opened pull request #259: Auth roles common validation and initialization cleanup (master...auth-roles-common-fix) http://git.io/OTHc2Q
* github-ff leaves
<pivotal-bot____>Benjamin Pennell added comment: "https://github.com/futures/fcrepo4/pull/259" https://www.pivotaltracker.com/story/show/65980058
Benjamin Pennell added comment: "https://github.com/futures/fcrepo4/pull/259" https://www.pivotaltracker.com/story/show/6597974414:31
Benjamin Pennell finished "AccessRoles session logout when post of new roles fails validation" https://www.pivotaltracker.com/story/show/65980058
Benjamin Pennell finished "AccessRolesTypes refactor to remove static flag checks and cleanup" https://www.pivotaltracker.com/story/show/65979744
* ermadmix joins14:33
* scossu leaves14:42
<benpennell>i seem to have somehow killed my local instance of fcrepo4 run by mvn jetty:run (infinispan CacheLoaderManager not starting), I'm guessing to start over i just need to delete fcrepo4-data in fcrepo-webapp?14:48
for anyone curious, the end of the stacktrace is "org.fusesource.leveldbjni.internal.NativeDB$DBException: Corruption: missing start of fragmented record(2)"14:51
<pivotal-bot____>Mike Daines added "Refactor ServletContainerAuthenticationProvider and AbstractRolesPEP so that principals are determined at the PEP layer" https://www.pivotaltracker.com/story/show/6661883015:02
Mike Daines edited "Refactor ServletContainerAuthenticationProvider and AbstractRolesPEP so that principals are determined at the PEP layer" https://www.pivotaltracker.com/story/show/6661883015:04
* ajs6f joins15:13
* ajs6f1 joins15:14
* ajs6f leaves
* ajs6f1 leaves15:17
* ajs6f joins15:18
gregjansen: can you review  https://github.com/futures/fcrepo4/pull/259 ?
<gregjansen>yes, sure thing15:19
* ajs6f1 joins15:29
* ajs6f leaves
<pivotal-bot____>A. "Torcrexilly" Soroka started "Improve event reporting facilities for node property changes" https://www.pivotaltracker.com/story/show/6660307215:48
* ajs6f1 leaves16:09
* scossu joins16:11
* ajs6f joins16:13
* scossu leaves16:15
<gregjansen>Well that is interesting. Looks like in some ITs the repository now deploys two times. By 2 refs to the same repo.xml.. (one via unit test runner, again via grizzly test container) I think in the old jcrfactory loading method, the same repository instance may have been returned, but using the modeshape engine, you get two different instances (and all sorts of lock file contention)16:20
<ajs6f>Wait, so it was _always_ deploying twice and we never noticed, somehow…? Or it just started to when you started using ModeshapEngine?16:21
<gregjansen>I have not confirmed that it was always happening16:26
<ajs6f>Okay. Can we reshape the Spring XML to avoid the issue?16:27
<gregjansen>It may be that the older factory method had a way of returning a singleton per repository config.
<ajs6f>It did. I wrote that.
It was intentionally written as a singleton.16:28
<gregjansen>looking into it. trying an approach where container just loads rest beans and the junit runner loads the repo.xml.. Not sure that beans from junit runner will make it into container context tho
Oh I thought I was talking about a modeshape factory
<ajs6f>No, the factory was ours.
<gregjansen>oh, was this some kind of protected scope thing?
<ajs6f>No, it was a plain singleton.16:29
I just couldn't imagine why we would want >1 JCR repo in play for a given Fedora instance
<gregjansen>yeah, me too. Are you talking about ModeShapeRepositoryFactoryBean? or the class that it was wrapping?
<ajs6f>ModeShapeRepositoryFactoryBean was was I wrote.16:30
That's where the singleton behavior was written.
Awesome name, eh? barmintor had fun ridiculing it.16:31
<gregjansen>okay, I see that.. well there were two instances of the same spring context loading I think,, repo.xml twice is what I see
<ajs6f>Okay, so that's bad Spring config on our part, no?
<gregjansen>they happen to have grabbed the same repo instance somehow, prior to my change.. now they start a second
<ajs6f>Well, it was clearly unstable.16:32
<gregjansen>yeah, but the tests rely on the bad config, so there is some refactoring there
<ajs6f>We were on borrowed time.
<gregjansen>np, have a look at my auth code
What's auth got to do with it?
<gregjansen>oh nothing
<ajs6f>Oh, I get it. You were being modest. {grin}16:33
<benpennell>this might be a dumb question, but is fedora supposed to keep objects around after you delete them or is benchtool not managing to clean up all the way after itself? (i just had to clean out 38gb of leftover test data)
<ajs6f>Fedora is most certainly not meant to keep stuff around after you get rid of it.
Oh, phooey.
What is it? Is it in the binary store directories?16:34
<gregjansen>is this with the jetty:run goal?
times a million
yes jetty:run
<ajs6f>grejansen: Is this familiar?
<benpennell>i suspect the 38gb directory might be why eclipse kept self destructing when i tried refreshing the project16:35
so it being in the binary-store directory and being run as jetty:run have some bearing or should i look around in benchtool?16:37
<ajs6f>benpennell: Not sure. We've had problems in the past with shutdowns from jetty:run, but I thought we had dealt with them16:41
Are you sure that deletions from the benchtool are being committed?16:42
Does the non-transactional benchtool show the same behavior?
<benpennell>i just ran a small test with benchtool and it left behind the same number of items i added without any restarts
i'm testing on the master version right now, wanted to make sure it wasn't something i introduced
fasseg: have you seen this?
<gregjansen>ajs6f: I think I will implement a context-aware singleton bean to support the injection of test beans from inside grizzly. Seems like the only way to go?16:44
<ajs6f>gregjansen: As long as we confine it to test situations, seems safe.16:45
<gregjansen>ajs6f: yeah, s'what I'm thinking too.
<ajs6f>gregjansen: Let's do it- I really don't want you to have to deal with this too much longer. We've got to get onto the clustering perf question.16:46
<gregjansen>I know it. This has really bogged me down this week. Fun troubleshooting, but time consuming16:47
<ajs6f>gregjansen: We had to go here, but I don't want to spend too much time here.
<gregjansen>ajs6f: okay, glad you think we had to go there too
<ajs6f>We know more now. We may leave behind some cleanup, but at least we know about it now.16:48
<benpennell>hm, according to fedora there are 0 objects in my repo, according to my harddrive there are 1016:51
<ajs6f>benpennell: Stop saying things like that.
Seriously, are you sure they are Fedora objects?
Are they JCR system objects?16:52
Are they things like the root node?
<benpennell>1.0M fcrepo4-data//fcrepo.binary-store-path/02/1e/95
1.0M fcrepo4-data//fcrepo.binary-store-path/02/1e
1.0M fcrepo4-data//fcrepo.binary-store-path/02
<ajs6f>Sure, but what's actually in them?
Are they versions?
It's okay for you to delete an object but there to be old versions left behind.16:53
<benpennell>maybe that's what's going on? I can create another ten by running benchtool again16:54
<ajs6f>Hm. Does benchtool exercise versioning?
<benpennell>i'll take a look16:55
Actually, we may need to look at that as an intenional parameter
Versioning may have effects on performance that we need to understand. But that's another set of tickets. :)
<pivotal-bot____>Mike Daines added "Implement the "metadata reader" role" https://www.pivotaltracker.com/story/show/6662771216:56
<gregjansen>I wonder if they could have been invalidated, but the files remain
I doubt that ModeShape has made us many promises in that regard, i.e. cleaning up files immediatly
<benpennell>is versioning on by default unless you tell it otherwise as part of the createObject request?16:57
<ajs6f>gregjansen: Definitely a possibility, but shold that happen via benchtool?
benpennell: Fair question: I don't know.
<gregjansen>ajs6f: depends more on the cache config than the client I would think
<ajs6f>gregjansen: MODE promises to clean up, but it's on a timer. It's a kind of GC>
cbeer investigated this many moons ago.16:58
It's possible that we shut down the repo before the GC cycle completes for the last objects.
(In benchtool, that is).
* daines1 leaves16:59
<benpennell>wonder what the cleanup timer is set to
* daines joins
<mikeAtUVa>ajs6f: Yeah?
<ajs6f>See benpennell's question ^^^
* daines leaves
<ajs6f>"is versioning on by default unless you tell it otherwise as part of the createObject request?"
<mikeAtUVa>ooh... ajs6f, benpennell: automatic version creation is not on by default... but I think the mix:versionable mixin is applied (meaning that versions can be made).17:00
<ajs6f>Hm. I think the binary GC cycle may be what we want to investigate here.
Anyways, I'm out for the day.17:01
benpennell: Let's definitely get into this tomorrow at standup.
<mikeAtUVa>ajs6f, benpennell: just having the mix:versionable mixin likely has some performance impact, though it can't be as big as actually creating versions.
<benpennell>mikeAtUVa: hm, so if i issue a plain old post to /rest/objects/<pid> to make a new object, it won't be versionable by default?
ajs6f: sure
<ajs6f>mikeAtUVa: The concern here is about benpennell finding binaries left behind after perf testing runs.
We're trying to figure out why.
<mikeAtUVa>benpennell: it will be versionable... but no new versions will be made until you explicitly make them.
<ajs6f>Okay, I'm seriously out of here. See y'all tomorrow.
* ajs6f leaves17:03
<benpennell>if i then delete the object, should a version be kept because it was versionable?
<mikeAtUVa>benpennell: Not sure... possibly... I'm not sure if a root version is created (and persisted) when a node is created.17:04
I'd try removing the mix:versionable mixin from the base CND.
<benpennell>okay, i'll try that
<mikeAtUVa>... or looking at the spec... depending on how you like your research.17:05
* scossu joins17:10
<benpennell>followed the instructions in https://wiki.duraspace.org/display/FF/How+to+disable+versioning and removed mixin:versionable from fedora, still getting leftover files. I'll do some debugging in benchtool to make sure the deletes are going through and then check on cleanup tomorrow17:18
* martind_ leaves17:48
* ermadmix leaves18:14
* dwilcox joins18:26
* dwilcox leaves18:28
* dwilcox joins
* dwilcox leaves18:32
* dwilcox joins18:47
* gregjansen leaves18:48
* gregjansen joins19:04
* dwilcox leaves19:17
* edInCo leaves19:18
* dwilcox joins19:27
* jbaiter leaves19:39
* jbaiter joins
* gregjansen leaves19:44
* jbaiter_ joins19:46
* jbaiter leaves19:47
* gregjansen joins19:51
* scossu leaves
* md5wz__ joins20:05
* mikeAtUVa leaves
* mikeAtUVa joins20:27
* cbeer leaves20:29
* md5wz__ leaves
* cbeer joins
* dwilcox leaves20:30
* dwilcox joins
* dwilcox leaves20:35
* scossu joins21:10
* scossu leaves21:15
* cbeer_ joins23:36
* cbeer leaves23:37
* ksclarke leaves00:28