Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* kaarefc joins03:10
* kaarefc leaves03:15
* mikeAtUVa leaves04:33
* mikeAtUVa joins04:34
* kaarefc joins06:30
<ermadmix>mikeAtUVa: We had a communication gap. My PR 20 and you PR 27 are trying to do the same thing. But your PR 27 covers all the PROPERTY related events, so it probably would be better to go with that.09:27
<mikeAtUVa>ermadmix: yeah, i was looking over those today. I don't have permissions to merge or commit to that project...09:30
* nbanks joins09:32
<ermadmix>mikeAtUVa: I guess I don't either... Are you able to even run tests on that project? I'm getting permGen errors.09:37
WARN 15:56:18.768 (DefaultMessageListenerContainer) Setup of JMS message listener invoker failed for destination 'topic://fedora' - trying to recover. Cause: PermGen space
tried some JVM tuning to no avail either:09:38
MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=1024m" mvn -Djacoco.agent.it.arg="-XX:MaxPermSize=1024m -Xmx1024m" -Djacoco.agent.ut.arg="-XX:MaxPermSize=256m -Xmx1024m" clean install
* nbanks leaves09:55
* nbanks joins09:56
<mikeAtUVa>ermadmix: is that when you try to build the fcrepo-jms-indexer-plugga ble09:58
* ksclarke leaves
<ermadmix>mikeAtUVa: yes
* ksclarke joins09:59
* ksclarke leaves10:00
<mikeAtUVa>For me, I've had success simply typing mvn clean install with the MAVEN_OPTS=-Xmx750M -XX:MaxPermSize=300M
<ermadmix>thanks - afk10:04
<pivotal-bot>Mike Durbin added comment: "This looks good so I'm "Delivering" it.10:10
I removed the bit about the fedora 3 federation connector... we can..." https://www.pivotaltracker.com/story/show/62257806
Mike Durbin delivered "Feature Documentation: Federation" https://www.pivotaltracker.com/story/show/62257806
* ksclarke joins10:12
<pivotal-bot>Mike Durbin delivered "A page to the Wiki describing recommend solutions to ingesting files of varying sizes." https://www.pivotaltracker.com/story/show/5506912610:13
* kaarefc leaves10:21
* fcrepo-bot joins10:22
* github-ff joins10:35
[fcrepo-jms-indexer-pluggable] mikedurbin opened pull request #28: Updated SolrIndexer to commit upon update. (master...commit-updates) http://git.io/E735jg
* github-ff leaves
<pivotal-bot>Mike Durbin finished "Property update messages don't result in updates by the jms-indexer-webapp." https://www.pivotaltracker.com/story/show/6280665410:36
Mike Durbin added comment: "https://github.com/futures/fcrepo-jms-indexer-pluggable/pull/2810:38
@awoods this might be fixed by a prior pull..." https://www.pivotaltracker.com/story/show/62806466
Mike Durbin added comment: "https://github.com/futures/fcrepo-jms-indexer-pluggable/pull/2710:39
@awoods these two pull requests indeed deal..." https://www.pivotaltracker.com/story/show/62806654
* github-ff joins11:03
[fcrepo-jms-indexer-pluggable] mikedurbin closed pull request #28: Updated SolrIndexer to commit upon update. (master...commit-updates) http://git.io/E735jg
* github-ff leaves
<pivotal-bot>Mike Durbin added comment: "@ajs6f indicated that this should be fixed in documentation, so I closed the pull request and will udpate th..." https://www.pivotaltracker.com/story/show/6280646611:04
Mike Durbin unstarted "org.fcrepo.indexer.solr.SolrIndexer doesn't commit on record updates." https://www.pivotaltracker.com/story/show/62806466
Mike Durbin started "org.fcrepo.indexer.solr.SolrIndexer doesn't commit on record updates." https://www.pivotaltracker.com/story/show/62806466
Nigel Banks accepted "A page to the Wiki describing recommend solutions to ingesting files of varying sizes." https://www.pivotaltracker.com/story/show/5506912611:25
* fcrepo-bot leaves11:26
<mikeAtUVa>cbeer, awoods, ermadmix, anyone who cares: the fcrepo-jms-indexer-pluggable expects an rdf:type of "http://fedora.info/definitions/v4/indexing#indexable"... Fedora 4 doesn't require that mixin type to actually exist, so would it be reasonable to dramatically simplify (by removing or burying information about registering such a mixin type) the documentation for setting up external indexing?11:28
Basically tell users to assert an RDF type on their nodes without specifying anywhere anything about that type except that it's a URI that's recognized by fcrep-jms-indexer-pluggable.11:30
<ermadmix>mikeAtUVa: SorryI don't understand your question. Rather than explain to users it is necessary to use the rdf:type http://fedora.info/definitions/v4/indexing#indexable to enable indexing, it would be simpler to what?11:39
<mikeAtUVa>Not tell them anything, because they don't have to register the namespace or the mixin type in order to get indexing to work.11:40
Er... sorry..
They still have to use the rdf:type, but they don't have to update node types and namespaces.11:41
ermadmix: I'm making minor tweaks to the External Search features page you wrote to reflect the things I've discovered.11:42
In general that page is good, it was certainly enough to get me to the place where the code worked... but I think I can simplify it and add some details for newbies like me to help them understand some of the concepts at a higher level.
<ermadmix>So have the indexing mixin come out of the box, not something to configure?11:44
<mikeAtUVa>ermadmix: I observed that even if there is no indexing mixin defined in the repository the code still works.11:45
ermadmix: I asked if anyone more knowledable about our best practices and RDF if it was a bad idea to rely on this behavior...
Also, there's a default indexing program, so I though I might move information about that mixin or about indexing program creation to an advanced page so there might be just 3 easy steps to get it working, then users can dig deeper to configure it. I have a short attention span, and if it takes too long before I can see results, sometimes I give up... and I wanted to update the documentation in a way to help people see results so11:48
* edInCo joins
<ermadmix>mikeAtUVa: Wait so is defining properties within mixins optional in general?
<mikeAtUVa>ermadmix: well, by default the basic node type allows all properties so you can still add that property...11:49
ermadmix: we'd lose the nofulltext and noqueryorder directives (unless they're the defaults)....11:50
ermadmix: to be clearer, on any node you can add the "indexing:hasIndexingTransformation" property as well as the type "indexing:indexable" even if you haven't registered the indexing:indexable mixin type.11:51
<ermadmix>mikeAtUVa - right, sorry that makes sense11:52
<mikeAtUVa>ermadmix: I still think we should talk about it... but since it's not a barrier to getting a proof-of-concept running, I'm thinking about moving that documentation to a sub-page referenced from a note.
* nbanks leaves11:55
<ermadmix>mikeAtUVa: Yeah, I have no objections to changing the documentation. But I found that "Define Indexing Namespace and Mixin" section self-helpful. Maybe that can go on a more general configuration page.11:56
<mikeAtUVa>ermadmix: yes, I found it useful too... I'll move it to another page and make my modifications then comment on the pivotal ticket and you can let me know if my changes are an improvement or not.11:57
<ermadmix>mikeAtUVa: will do11:58
<mikeAtUVa>ermadmix: thanks, I'll let you know when it's ready...12:00
<awoods>mikeAtUVa: There seem to be a lot of tickets still undelivered in pivotal...
mikeAtUVa: I guess folks were unable to review them yesterday?12:01
<mikeAtUVa>awoods; Should I deliver my own?
<awoods>mikeAtUVa: Based on yesterday's call, I was expecting the "requesters" to review tickets.
<mikeAtUVa>awoods: I'm working on external search now... there's one simple PR that needs to be merged...
<pivotal-bot>Mike Durbin delivered "Create "Documentation DRAFT > Feature Walkthrough" wiki page." https://www.pivotaltracker.com/story/show/6218834212:02
<awoods>mikeAtUVa: ok. Do you have any sense of the other outstanding/un-reviewed tickets?
<mikeAtUVa>awoods: I don't know about franks stuff.... there's that heap one that I think only Chris observed and can close...12:03
<pivotal-bot>Andrew Woods unstarted "fcrepo-jms-indexer should maintain its own broker for robustness" https://www.pivotaltracker.com/story/show/61414712
<awoods>mikeAtUVa: It would be nice if we could pull in the PRs from: https://www.pivotaltracker.com/story/show/6226251212:04
<pivotal-bot>feature: Unit tests for indexer framework (started) / owner: A. "Horbulaco" Soroka
<awoods>mikeAtUVa: ...although ajs6f has not claimed that ticket to be finished.
<pivotal-bot>Andrew Woods unstarted "More integration tests for the named-fields indexer gear" https://www.pivotaltracker.com/story/show/6226275812:05
<mikeAtUVa>If you give me commit privileges I'd be happy to look at stuff on that project.
<pivotal-bot>Andrew Woods unstarted "Work out the cause of the large file ingest test failure with rhauch" https://www.pivotaltracker.com/story/show/6141839812:06
<mikeAtUVa>awoods: I don't have a huge amount of time, but my goal today is to deliver ermadmix's documentation on external search with some modifications to reflect my experience...
<awoods>mikeAtUVa: what about: https://www.pivotaltracker.com/story/show/6280646612:07
<pivotal-bot>bug: org.fcrepo.indexer.solr.SolrIndexer doesn't commit on record updates. (started) / owner: Mike Durbin
<mikeAtUVa>awoods: I think it's likely... I *did* get the solr stuff to work and I understand the pieces... though we need to fix that property bug (PR https://github.com/futures/fcrepo-jms-indexer-pluggable/pull/27
awoods: adam informed me that that's by design, so I'll finish it by updating our documenation for a solr Quick guide (tell them to set autocommit)12:08
<awoods>mikeAtUVa: I just permgen'd on building fcrepo-jms-indexer-pluggable, as well. ...will try again with the suggested MAVEN_OPTS12:11
mikeAtUVa/ermadmix: It is interesting, when building fcrepo-jms-indexer-pluggable, these MAVEN_OPTS work:12:19
-Xmx750M -XX:MaxPermSize=300M
but these do not:
-Xmx1024m -XX:MaxPermSize=1024m
<pivotal-bot>Andrew Woods accepted "Unit tests for indexer framework" https://www.pivotaltracker.com/story/show/6226251212:20
* ksclarke leaves12:21
<pivotal-bot>Andrew Woods added comment: "@md, this PR looks good... although an accompanying unit test should exist. @escowles, you mentioned concer..." https://www.pivotaltracker.com/story/show/6280665412:24
Andrew Woods edited "Feature Documentation: External Search" https://www.pivotaltracker.com/story/show/6225811412:25
<awoods>mikeAtUVa: it would also appear that jenkins has not been building jms-indexer-pluggable lately. http://ci.fcrepo.org/jenkins/view/FF/job/fcrepo-jms-indexer-pluggable/12:26
cbeer: Can you give this ticket a quick look: https://www.pivotaltracker.com/story/show/6148261012:28
<pivotal-bot>bug: Fix o.f.k.o.SimpleObserver so it doesn't blow the heap out with large volumes of objects (finished) / owner: Benjamin Armintor
Andrew Woods edited "Create "Documentation DRAFT > Feature Walkthrough" wiki page." https://www.pivotaltracker.com/story/show/6218834212:29
Andrew Woods delivered "Feature Documentation: External Search" https://www.pivotaltracker.com/story/show/6225811412:30
* ksclarke joins12:44
<pivotal-bot>Esme Cowles added comment: "It's possible to have slashes in property names -- for example, we have some properties like "http://fedora...." https://www.pivotaltracker.com/story/show/6280665412:56
Andrew Woods added comment: "@escowles, could you please create a ticket to have the event generation publish the node path and properti..." https://www.pivotaltracker.com/story/show/6280665413:30
Esme Cowles added "Property update events should have separate node and property fields" https://www.pivotaltracker.com/story/show/6285448813:34
<mikeAtUVa>Is the plan to try to get that in the release or use my broken-but-better-than-nothing patch to the indexer and fix it later?13:37
<awoods>mikeAtUVa: are you asking about the unit test?13:38
<mikeAtUVa>no, about escowles new ticket
awoods: I'm writing the unit test now.
<awoods>mikeAtUVa: escowles' new ticket is for next sprint.13:39
<mikeAtUVa>ok, cool
<ermadmix>awoods: re [12:19] comment - yes got the same, stupid question is M and m the same?13:59
<pivotal-bot>Eric James added comment: "One thing I noticed is that if you change >1 property, you just get just 1 PROPERTY_CHANGE event on just one ..." https://www.pivotaltracker.com/story/show/6285448814:33
Mike Durbin added comment: "I added a unit test for the IndexerGroup class. There were no tests, now there's 85% line coverage." https://www.pivotaltracker.com/story/show/6280665415:49
* ksclarke leaves16:12
<pivotal-bot>Mike Durbin added comment: "@ericjames If you think the reorganization is acceptable, feel free to "Accept" this ticket. If you feel I ..." https://www.pivotaltracker.com/story/show/6225811416:14
* ksclarke joins16:23
<pivotal-bot>Eric James added "inconsistent behavior adding node" https://www.pivotaltracker.com/story/show/6286760417:07
Eric James edited "inconsistent behavior adding node" https://www.pivotaltracker.com/story/show/6286760417:08
Mike Durbin finished "org.fcrepo.indexer.solr.SolrIndexer doesn't commit on record updates." https://www.pivotaltracker.com/story/show/6280646617:09
Mike Durbin added comment: "This is documented in the Solr Indexing Quick Guide: ""
https://wiki.duraspace.org/display/FF/Solr+Indexing+Qui..." https://www.pivotaltracker.com/story/show/62806466
* mikeAtUVa leaves
* ermadmix leaves17:13
* edInCo leaves19:09
<awoods>ermadmix: I believe the java-opts are not case-sensitive.19:21