Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* mikeAtUVa joins08:41
* martinjd leaves08:54
* ksclarke_idle joins09:08
awoods: did you remove the contact ModeShape ticket? or what does setting level of effort to 0 do? I don't seem to be able to find it after I did that (in prep for marking it not necessary)09:33
<awoods>ksclarke_idle: I removed the ticket, since it was not needed.09:34
<ksclarke_idle>okay, great
<awoods>ksclarke_idle: Any news on the 500,000 item run?
<ksclarke_idle>yes, I tried four times on two different boxes but got exceptions each time (during the prep stages); it was not obvious to me what the exceptions were related to so I didn't delve too deeply09:36
but I figured that would be another ticket (if needed0
I saved one of the exceptions is someone who is more familiar with the code base wanted to look (and might have a sense of what the cause was)
in case someone who... that is09:37
<awoods>ksclarke_idle: maybe create a ticket and attach the log output / exception
<ksclarke_idle>ok, I'll try once more today to get the log output too
* escowles_idle joins09:40
* longshou joins10:39
* scossu joins11:11
<awoods>longshou: What is your github user handle?11:39
escowles ^^ do you know?11:40
* cbeer joins11:43
* pivotal-bot joins11:55
Andrew Woods added comment: "You should have write access to the above github repository now, @longshous." https://www.pivotaltracker.com/story/show/6919058811:57
* longshou leaves11:58
* edInCo joins
<ksclarke>awoods: when you say in the writer ticket: "deploy f4 from the authz war"; I should be able to drop fcrepo-webapp/target/fcrepo-webapp-4.0.0-alpha-5-SNAPSHOT-auth.war into tomcat, start it, and it should run?12:04
<awoods>ksclarke: you will also need to update your tomcat-users.xml12:05
ksclarke: https://wiki.duraspace.org/display/FF/How+to+Configure+Servlet+Container+Authentication#HowtoConfigureServletContainerAuthentication-Tomcat12:06
<escowles>awoods: longshou's github account is lsitu12:15
* scossu leaves
<awoods>escowles: got it. thanks
* scossu joins12:29
<pivotal-bot>Chris Beer added "Add support for HTTP OPTIONS verb" https://www.pivotaltracker.com/story/show/6946750013:03
* ksclarke scratches head13:04
<pivotal-bot>Chris Beer added "Support the HTTP `prefer` header" https://www.pivotaltracker.com/story/show/6946774213:07
Chris Beer edited "Support the HTTP `prefer` header" https://www.pivotaltracker.com/story/show/69467742
Chris Beer added "Define fcrepo4-specific Prefer header values for adding additional sources of triples " https://www.pivotaltracker.com/story/show/6946851013:12
Chris Beer added "Use HTTP Link headers for LDP containers" https://www.pivotaltracker.com/story/show/6946922613:19
Chris Beer added "Implement new style of membership predicates" https://www.pivotaltracker.com/story/show/6946931213:20
Kevin Clarke added comment: "To get the authz war to run: ""
1) Copy from rcrepo4/fcrepo-webapp/target/fcrepo-webapp-4.0.0-alpha-5-SNAPSHO..." https://www.pivotaltracker.com/story/show/68547530
* benpennell1 leaves
<pivotal-bot>Kevin Clarke added comment: "without step 2 (number 2... err, 3) you get a bunch of Spring errors" https://www.pivotaltracker.com/story/show/6854753013:21
<ksclarke>hmm, so you can't edit a comment in pivotal?
<awoods>correct
<ksclarke>yuck13:22
<awoods>ksclarke: https://wiki.duraspace.org/display/FF/Application+Configuration "Configuration Elements" for point #2/3
<pivotal-bot>Chris Beer added "Support LDP Paging spec" https://www.pivotaltracker.com/story/show/6946962813:23
<ksclarke>I've been spoiled by jetty:run
there must be a default config/property in there for it with that13:24
<awoods>on a call13:25
<pivotal-bot>Chris Beer added "fcr:content responses should include an HTTP Link header back to the "datastream"" https://www.pivotaltracker.com/story/show/69469938
Chris Beer added comment: "Here's a relevant note in the LDP spec: ""13:29
Non-normative note: Clients might provide properties equivalent to t..." https://www.pivotaltracker.com/story/show/66168308
Chris Beer edited "Including "<> a <ldp:Container >" triple in a PUT request body should not result in a 500 status code and stack trace" https://www.pivotaltracker.com/story/show/66168308
Chris Beer added "Publish restrictions on LDP creation" https://www.pivotaltracker.com/story/show/6947074013:32
Chris Beer added "Add fcrepo4 integration tests based on LDP use cases and requirements" https://www.pivotaltracker.com/story/show/6947140613:39
Chris Beer added comment: "Duplicate of #54292904" https://www.pivotaltracker.com/story/show/69467500
Chris Beer added comment: "I think it could make sense as an HTTP header of some sort.. I wonder if Last-Modified is. too much of a stretch" https://www.pivotaltracker.com/story/show/6830156213:40
Chris Beer edited "Return timestamp when creating node" https://www.pivotaltracker.com/story/show/6830156213:41
Esme Cowles started "Use HTTP Link headers for LDP containers" https://www.pivotaltracker.com/story/show/6946922613:50
<awoods>ksclarke: The point before being that you should be able to provide JAVA_OPTS that specify the location of the fcrepo4-data directory for Tomcat.13:55
<ksclarke>yeah, I saw that after you pointed it out13:56
would be nice to be able to edit the comment but it was mostly for my own use
<awoods>ksclarke: btw, "editing" pivotal comments is a copy/modify/paste/delete process.
* github-ff joins
[fcrepo4] lsitu opened pull request #284: Reduce the console logging when running the one-click-run application. (master...feature/jetty_logging) http://git.io/OuLAKw
* github-ff leaves
<ksclarke>ah, okay
good idea13:57
<pivotal-bot>Longshou Situ added comment: "https://github.com/futures/fcrepo4/pull/284" https://www.pivotaltracker.com/story/show/63493246
Longshou Situ finished "Reduce "One-click-run" Logging" https://www.pivotaltracker.com/story/show/63493246
<ksclarke>awoods: unrelated, was also curious about a comment from that link about jetty-maven-plugin being used for testing only; I know that's a common belief but are you aware of any place that documents concrete reasons for that? (e.g., lacks init.d script, has XYZ security problems, etc.)13:58
I know even jetty devs say that but I've never seen reasons for it that were sort of exercises in hand-waving13:59
<awoods>ksclarke: let me find what you are referring to...
<ksclarke>that weren't, that is
<awoods>ksclarke: Are you asking if jetty:run is ever used for production installations?14:01
and if not, why.
<ksclarke>I'm asking whether you've ever seen documentation (with concrete reasons) why it shouldn't be
I actually wonder this off and on, but seeing the comment on the page prompted me to ask :-)14:02
(off and on over the period of years, not just related to fcrepo4)14:03
<awoods>ksclarke: which comment on which page?
ksclarke: It seems like there are two questions: one regarding jetty:run and the other involving embedded jetty.14:04
<ksclarke>https://wiki.duraspace.org/display/FF/Application+Configuration (under deployments... the last two are "for testing")
I'm specifically interested in the jetty-maven-plugin use, not embedded jetty
(or why for embedded would be recommended when the plugin isn't)
<awoods>ksclarke: Generally, I suspect you would want to separate your build environment from your production env.14:05
<ksclarke>the jetty devs will say the plugin isn't intended for production but then recommend embedding as an option
<awoods>ksclarke: jetty:run monitors the source directory for updates.
<ksclarke>but that's an _advantage_ to the plugin... it can hook right into Maven profiles :-)
you have a maven prod profile, dev profile, stage profile, etc.14:06
<awoods>ksclarke: and as far as I know, you can configure the jetty in jetty:run with whichever options you could want.
for a single server install.14:07
<ksclarke>I don't mean to pressure you on the topic, it's just always struck me as interesting since it's repeated as best practice14:08
<awoods>ksclarke: It is an interesting question. Maybe it is old-school to feel the need to separate the application from the container.
<pivotal-bot>Andrew Woods edited "ExecutionError while trying to ingest 500k records" https://www.pivotaltracker.com/story/show/6945415214:09
<ksclarke>I wonder with our changing environments whether that's a big part of it
interested to hear solr is going more towards the stand alone application (rather than app in container route) soon14:10
anyway, back to work :-)14:11
<awoods>ksclarke: https://wiki.duraspace.org/display/FF/Test+-+Platform+Profile%3A+ksclarke+EC2+m3.medium14:12
was the EBS set up with provisioned IOPS?
ksclarke: or did you just measure the average performance?
<ksclarke>no, standard.. just pulled that from an info page on aws14:13
<awoods>thanks
<pivotal-bot>Andrew Woods added comment: "@ksclarke, can you throw a summary/conclusion at the top of this wiki page? Then we can call it good-to-go...." https://www.pivotaltracker.com/story/show/6866296414:14
Andrew Woods rejected "Execute cross-backend performance testing regime" https://www.pivotaltracker.com/story/show/68662964
<ksclarke>I did upgrade the allocated storage to 650GB too but I don't know if it's worth updating for that14:15
* scossu leaves14:17
<pivotal-bot>Kevin Clarke added comment: "Notes to myself: ""14:18
1) Move jar from fcrepo-webapp/target/fcrepo-webapp-4.0.0-alpha-5-SNAPSHOT-auth.war to to..." https://www.pivotaltracker.com/story/show/68547530
* longshou joins14:19
<pivotal-bot>Kevin Clarke added comment: "Just to close the circle, I did try to run a 500,000 object test but did not succeed. This ticket is a res..." https://www.pivotaltracker.com/story/show/6866296414:20
* scossu joins14:24
* github-ff joins
[fcrepo4] awoods pushed 1 new commit to master: http://git.io/Kyl0hw
fcrepo4/master 6c9cdca ajs6f: Create new identifier translation framework with unit tests...
* github-ff leaves
<pivotal-bot>Andrew Woods added comment: "Resolved with: https://github.com/futures/fcrepo4/commit/6c9cdcaba6c4c9d5f5d687d97a12eed20a4e0b8d" https://www.pivotaltracker.com/story/show/6761836414:25
Andrew Woods delivered "Implement identifier translation framework" https://www.pivotaltracker.com/story/show/67618364
* github-ff joins14:26
[fcrepo4] awoods closed pull request #275: Hierarchy translator (master...HierarchyTranslator) http://git.io/ExGChA
* github-ff leaves
<pivotal-bot>Kevin Clarke started "Execute cross-backend performance testing regime" https://www.pivotaltracker.com/story/show/6866296414:36
* ksclarke is still curious about running 60k object tests just because of the slight uptick in the 2mb runs14:39
<pivotal-bot>Esme Cowles added comment: "We currently have just Link: http://www.w3.org/ns/ldp#Resource;rel="type", so I've added the DirectContainer..." https://www.pivotaltracker.com/story/show/69469226
<cbeer>awoods: is there a new version of the Fedora Commons intellij license?14:41
<pivotal-bot>Kevin Clarke finished "Execute cross-backend performance testing regime" https://www.pivotaltracker.com/story/show/6866296414:42
* github-ff joins14:44
[fcrepo4] escowles created ldp-updates (+1 new commit): http://git.io/V4rrjA
fcrepo4/ldp-updates 2e8098e Esmé Cowles: Adding Link header referencing LDP#DirectContainer (fixes https://www.pivotaltracker.com/s/projects/684825/stories/69469226)
* github-ff leaves
<awoods>cbeer: my license expires Sept 2, 201414:49
<cbeer>awoods: hm. the latest i have is from eddies expiring Feb 2014.
<awoods>ksclarke: feel free to continue with your ticket for 60,000 objects.
<pivotal-bot>Esme Cowles started "Implement new style of membership predicates" https://www.pivotaltracker.com/story/show/69469312
<ksclarke>I will
awoods ^
<awoods>cbeer: check inbox14:50
<cbeer>awoods: thanks.
<awoods>I am stepping out for a moment.
* travis-ci joins14:53
[travis-ci] futures/fcrepo4#1715 (master - 6c9cdca : ajs6f): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/compare/95a825ef5e1d...6c9cdcaba6c4
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/22974506
* travis-ci leaves
<pivotal-bot>Chris Beer estimated "fcr:content responses should include an HTTP Link header back to the "datastream"" as 1 point https://www.pivotaltracker.com/story/show/69469938
Chris Beer started "fcr:content responses should include an HTTP Link header back to the "datastream"" https://www.pivotaltracker.com/story/show/69469938
* tecoripa joins15:18
* tecoripa leaves
<ksclarke>so is the "Access Roles" update box on the lower right of the webui not intended to display what's currently configured? just provide a way to update from the webui?15:22
* travis-ci joins
[travis-ci] futures/fcrepo4#1716 (ldp-updates - 2e8098e : Esmé Cowles): The build has errored.
[travis-ci] Change view : https://github.com/futures/fcrepo4/commit/2e8098e97e3d
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/22975932
* travis-ci leaves
* gregjansen joins15:27
<pivotal-bot>Andrew Woods accepted "Implement identifier translation framework" https://www.pivotaltracker.com/story/show/6761836416:35
Longshou Situ added comment: "Wiki (https://wiki.duraspace.org/display/FF/Indexer+Configuration) now updated for the support of multiple..." https://www.pivotaltracker.com/story/show/68853354
<awoods>ksclarke: yes
* ksclarke nods
<awoods>ksclarke: and: https://www.pivotaltracker.com/story/show/6317822816:36
<pivotal-bot>Longshou Situ finished "Multiple Fedoras Feeding Single Triplestore" https://www.pivotaltracker.com/story/show/68853354
feature: Only enable AuthZ UI form when applicable (unscheduled) / owner:
* github-ff joins16:38
[fcrepo4] escowles pushed 1 new commit to ldp-updates: http://git.io/NgdAzw
fcrepo4/ldp-updates 6106b93 Esmé Cowles: Updating LDP membership triples (fixes https://www.pivotaltracker.com/story/show/69469312)
* github-ff leaves
<pivotal-bot>feature: Implement new style of membership predicates (started) / owner: Esme Cowles
* github-ff joins16:40
[fcrepo4] escowles opened pull request #285: LDP Updates (master...ldp-updates) http://git.io/7aC_Hg
* github-ff leaves
<pivotal-bot>Esme Cowles added comment: "https://github.com/futures/fcrepo4/pull/285" https://www.pivotaltracker.com/story/show/6946922616:41
Esme Cowles finished "Use HTTP Link headers for LDP containers" https://www.pivotaltracker.com/story/show/69469226
Esme Cowles added comment: "https://github.com/futures/fcrepo4/pull/285" https://www.pivotaltracker.com/story/show/69469312
Esme Cowles finished "Implement new style of membership predicates" https://www.pivotaltracker.com/story/show/69469312
Longshou Situ added comment: "@awoods @escowles I am now asking for permissions to include some datasets from Wagner and SIO Gelogical c..." https://www.pivotaltracker.com/story/show/6919058816:50
* mikeAtUVa leaves16:51
<pivotal-bot>Andrew Woods added comment: "If it is for demonstration purposes, 10 - 20 records is probably a good start. dc:title and dc:subject are ..." https://www.pivotaltracker.com/story/show/6919058816:56
* github-ff joins17:02
[fcrepo4] cbeer created described-by (+1 new commit): http://git.io/ag32Cw
fcrepo4/described-by 8eefe10 Chris Beer: add HTTP Link headers for describedby/describes to link a fcr:content resource to its metadata
* github-ff leaves
* github-ff joins17:03
[fcrepo4] cbeer opened pull request #286: add HTTP Link headers for describedby/describes to link a fcr:content resource to its metadata (master...described-by) http://git.io/dF-7IQ
* github-ff leaves
<pivotal-bot>Chris Beer added comment: "https://github.com/futures/fcrepo4/pull/286" https://www.pivotaltracker.com/story/show/69469938
Chris Beer finished "fcr:content responses should include an HTTP Link header back to the "datastream"" https://www.pivotaltracker.com/story/show/6946993817:04
Longshou Situ added comment: "The datasets has its own complex component hierarchy, and we can construct some simple objects in a collec..." https://www.pivotaltracker.com/story/show/6919058817:05
Chris Beer started "Support the HTTP `prefer` header" https://www.pivotaltracker.com/story/show/69467742
Esme Cowles added comment: "Some of the sample objects we may be able to contribute are the most-complex objects in our repository, whic..." https://www.pivotaltracker.com/story/show/6919058817:07
<cbeer>escowles: fcrepo4 hasn't done anything with async request handling yet, right?17:15
<awoods>cbeer: right17:17
cbeer: waiting on jersey217:18
<cbeer>awoods: ok. LDP is now referring to a spec that might deal with our HTTP API semantics for us
http://tools.ietf.org/html/draft-snell-http-prefer-18#page-9
<awoods>cbeer: that is a start. Although "...whether and how any given server supports asynchronous17:20
responses is an implementation specific detail that is considered to
be out of the scope of this specification."
* gregjansen leaves17:21
<cbeer>and rightly so.
<awoods>cbeer: the point being that I do not see the expected sequence of interactions after the initial: POST/Prefer17:24
<cbeer>awoods: right. that's not the point of the prefer header, just one use case.17:25
<awoods>cbeer: ok. We will still need to define the Fedora sequence from the initial request on.17:26
* travis-ci joins17:36
[travis-ci] futures/fcrepo4#1719 (described-by - 8eefe10 : Chris Beer): The build passed.
[travis-ci] Change view : https://github.com/futures/fcrepo4/commit/8eefe101b388
[travis-ci] Build details : http://travis-ci.org/futures/fcrepo4/builds/22987786
* travis-ci leaves
* nikhiltri joins18:26
awoods: you there?18:27
<awoods>hello, nikhiltri
nikhiltri: where is the issue?
* edInCo leaves18:28
* github-ff joins
[fcrepo4] awoods deleted HierarchyTranslator at ef923a1: http://git.io/YxZHpA
* github-ff leaves
<nikhiltri>Hey, so I've got the credentials set in my maven settings.xml, I'm setting them as system properties using properties-maven-plugin, but when a deploy the way, the webapp breaks while loading the the Spring config saying the credential values haven't been set.18:29
<awoods>nikhiltri: The issue is probably in either your pom.xml or the indexer-core.xml18:30
nikhiltri: If you commit your changes to your existing branch, I can take a look.18:31
* ksclarke leaves18:33
<nikhiltri>They're up: https://github.com/nikhiltri/fcrepo-jms-indexer-pluggable/blob/master/fcrepo-jms-indexer-webapp/pom.xml#L9418:34
https://github.com/nikhiltri/fcrepo-jms-indexer-pluggable/blob/master/fcrepo-jms-indexer-webapp/src/main/resources/spring/indexer-core.xml#L58
<awoods>nikhiltri: The property you are setting here: https://github.com/nikhiltri/fcrepo-jms-indexer-pluggable/blob/master/fcrepo-jms-indexer-webapp/pom.xml#L10618:36
"fcrepo.username"
Does not match the property name you are looking for here: https://github.com/nikhiltri/fcrepo-jms-indexer-pluggable/blob/master/fcrepo-jms-indexer-webapp/src/main/resources/spring/indexer-core.xml#L58
"fedoraUsername"
nevermind
nikhiltri: I assume in your settings.xml, you are using: "fedora.repo.username"?18:37
<nikhiltri>Yes, and the file is at fcrepo-jms-indexer-webapp/conf/settings.xml18:38
<awoods>nikhiltri: ??18:40
<nikhiltri>awoods: Yes, and the file is at fcrepo-jms-indexer-webapp/conf/settings.xml
<awoods>nikhiltri: the settings.xml is either in your home directory, or in the maven install directory by default
<nikhiltri>Ahh
<awoods>nikhiltri: If you have the settings.xml somewhere else, you need to tell maven that...
nikhiltri: with: "mvn -s <wherever>/settings.xml" jetty:run18:41
<nikhiltri>awoods: I'll give that a try.18:42
<awoods>nikhiltri: but normally the settings.xml is in one of the first two places that I mentioned. It tends to hold secret information that you do not want to let out.
<nikhiltri>awoods: Yes, makes sense now.
<awoods>nikhiltri: see: https://maven.apache.org/settings.html
"Quick Overview"18:43
<nikhiltri>I saw that, but mistook $M2_HOME for the directory of my codebase, my mistake.
<awoods>nikhiltri: also, you will want to put a default value here: https://github.com/nikhiltri/fcrepo-jms-indexer-pluggable/blob/master/fcrepo-jms-indexer-webapp/src/main/resources/spring/indexer-core.xml#L5818:44
<nikhiltri>Yea, I realized that in this conversation, will do.18:45
<awoods>nikhiltri: or else the user will will see the error you are seeing in the case that they are not using any authZ on fcrepo
<nikhiltri>awoods: In my settings.xml, I should be setting my properties in <profiles> <profile> <properties>, yea? #sanitycheck18:51
<awoods>nikhiltri: yes, and that <profile> should be active:18:53
nikhiltri: such as: <profile>
<id>fcrepo</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
...
<nikhiltri>awoods: Something's still not wiring up properly, when I copy the war to a Tomcat server and start it up, I get the error: Could not resolve placeholder 'fcrepo.username' in string value "${fcrepo.username}"19:11
<awoods>nikhiltri: did it work with jetty:run?19:13
<nikhiltri>awoods: it seems to, though I'm not seeing the log messages I expect. I'm not used to the jetty enviro, though, so I could be missing something.19:17
<awoods>nikhiltri: are you fairly certain that the correct settings.xml is being picked up?
nikhiltri: run mvn with the -X option (mvn -X clean)19:19
nikhiltri: it will tell you which settings.xml is/are being used.
nikhiltri: are you on a mac or linux machine?
<nikhiltri>awoods: mac19:21
awoods: Looks like it is picking up the settings file: [DEBUG] Reading user settings from /Users/ntrive/.m2/settings.xml19:23
<awoods>nikhiltri: I am building your branch now.19:28
* scossu leaves19:31
* github-ff joins19:38
[fcrepo-jms-indexer-pluggable] nikhiltri opened pull request #36: Added authenticated access to Fedora repo. (master...master) http://git.io/HMp-XQ
* github-ff leaves
<awoods>nikhiltri: I pulled down your code base when I mentioned it before. I stood up fuseki and fcrepo. I started the jms-indexer (jetty-run) with a settings.xml that included: <fedora.repo.username>fedoraAdmin</fedora.repo.username>20:07
<fedora.repo.password>secret3</fedora.repo.password>. I put the debugger on the jms-indexer and saw that the fedora username and password were set correctly. However, the basic-auth machinery was not working and a 401 response was seen with repository updates.
nikhiltri: you should be able to do the same thing with tomcat, but you will have to set the system properties in the tomcat start-up config files.20:08
<nikhiltri>Yea, I just saw that. It looks like the wiring of the settings are working properly when I start the indexer up in Jetty, but get list when I copy the war over to Tomcat.
<awoods>nikhiltri: yes, in tomcat you will need to set the system properties via "JAVA_OPTS=... -Dfcrepo.username=whatever"20:09
<nikhiltri>awoods: Ahh, ok, I'll check that out. I regressed my code from httpclient 4.3.x to 4.2.x, and this all came up while I was trying to test it. I'll get the basic0auth working in my next commit.
awoods: That's the missing piece for Tomcat, makes sense. Thanks!
* longshou leaves21:08
* ksclarke joins21:20
* nikhiltri leaves21:54