Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* dhlamb joins06:17
* dwilcox joins07:11
* dhlamb leaves07:18
* dwilcox leaves07:24
* dwilcox joins07:37
* awoods joins07:54
* coblej joins08:19
* dwilcox leaves08:36
* dwilcox joins08:37
* dwilcox leaves08:38
* dwilcox joins
* dwilcox leaves
* dwilcox joins08:39
* dwilcox leaves
* dwilcox joins08:40
* dwilcox leaves
* coblej leaves
* lsitu joins08:50
* coblej joins08:53
* youn joins08:58
<awoods>https://wiki.duraspace.org/display/FF/2017-05+Import+-+Export+Sprint+04+Meetings09:01
* benpennell joins
* informatician joins09:03
I'm here having trouble with my mic.09:04
<-- Kieran
* escowles joins
<informatician>Proceed! I have another mic that I'm going to use instead, need to go grab it. Brb.
* Bridget joins09:05
* mikeAtUVa joins09:17
<benpennell>*silence*09:21
<escowles>i said it would be good to have a meeting tomorrow or weds, to give people a chance to ask questions and get things resolved
* dbernstein joins09:22
<coblej>sound seems to be cutting in and out for me
<benpennell>is everyone cutting out or just esme?
<Bridget>I'm losing much of the sound too
<escowles>sorry, sounds like it's me...09:23
<benpennell>i think it might be everyone
<mikeAtUVa>I can hear ok using the web dialer.
<benpennell>and me
i'm connected via phone09:24
<dbernstein>me too.
<escowles>i'm using skype on my computer
<benpennell>i don't have headphones with me since i was just using my phone...
<awoods>https://plus.google.com/hangouts/_/fedora-commons.org/f4-import?hceid=ZmVkb3JhLWNvbW1vbnMub3JnXzhtcmVjdnIzZzZzMnNtMmZkdWV0Zm9yNjlzQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20.bm4b9e53su6295em2l4nc32cfc&authuser=0
* westgard joins09:25
<escowles>awoods: it says "Requesting to join the video call..."
<awoods>youn? benpennell? will you be able to join?09:26
* dhlamb joins09:27
<youn>yes, I have to install the plugin09:28
<benpennell>trying
* youn leaves09:29
* youn joins
* yamil joins09:32
* youn_ joins
<benpennell>10am wednesday and whatever time tuesday are good with me
* youn leaves09:34
<mikeAtUVa>It's all good here.09:35
* youn joins09:38
* youn_ leaves09:39
<Bridget>is it possible to get added to the wiki page for the sprint, links to where we can should get the latest import/export tool and which version of Fedora we should be using for testing?09:40
* peichman joins
* yamil leaves09:50
<escowles>Bridget: the most recent release is at: https://github.com/fcrepo4/fcrepo4/releases/tag/fcrepo-4.7.209:53
* peichman leaves
<escowles>the first file (fcrepo-webapp-4.7.2-jetty-console.jar) is the one-click jar file that's the easiest to run without having to setup tomcat etc.09:54
the import/export tool: https://github.com/fcrepo4-labs/fcrepo-import-export/releases/tag/fcrepo-import-export-0.1.009:56
* peichman joins09:57
* yamil joins09:59
<dbernstein>Verification tool : https://github.com/fcrepo4-labs/fcrepo-import-export-verify10:02
<mikeAtUVa>I have access to AP Trust (system and staff).10:04
<dbernstein>https://github.com/fcrepo4-labs/fcrepo-sample-dataset10:22
<benpennell>sorry i have to bail out for my search committee now :(10:29
* benpennell leaves
* yamil leaves10:30
* benpennell joins10:31
* benpennell leaves10:35
<Bridget>have to hop off. I'll try to have my environment setup and some initial imports tested by check-in on wed.10:40
* Bridget leaves
<escowles>lsitu: i think creating an empty binary is a good idea, because it would let you preserve the metadata — the other option would be just not creating the binary or metadata10:45
* yamil joins10:55
* benpennell joins10:58
* umgrosscol joins11:03
* informatician leaves11:05
* ksclarke leaves
* peichman leaves11:06
<escowles>lsitu: i divided up the bug tickets — go ahead and work on the one that looks best to you, and we can always reassign later if one of us runs out11:07
* github-ff joins
[fcrepo-specification] awoods closed pull request #114: Use case template (master...use-case-template) https://git.io/v91jb
* github-ff leaves
* ksclarke joins11:08
* peichman joins11:10
* benpennell leaves
<lsitu>escowles: It sounds good. Thanks.11:11
What the other option to not the other option would be just not creating the binary or metadata? Do you mean just crate an empty resource? It looks like Fedora is complaining about misiing resource reference for the binary files during import.
<escowles>yes, the other option would be to create an empty binary so the links to it would not cause errors11:12
<lsitu>Okay. I’ll create the empty binary then. Thanks.11:13
<escowles>lsitu: the main problem i see with that is that the empty binary would have a different checksum than what's in the real binary, so i think it will lead to validation problems11:18
so i think when the importer can't find a binary, it shouldn't import links to the binary either
<lsitu>escowles: Hmm, yep, that makes it troublesome.11:20
Will it work if we just create an empty resource?11:22
<escowles>i don't think so — if we create an empty resource it will have a bad checksum, not the same checksum as the original file11:23
so i think either not exporting any references to the binaries, or not importing them, will avoid that issue
* dwilcox joins11:24
* dwilcox leaves11:25
* dwilcox joins
* dwilcox leaves11:26
* dwilcox joins11:27
* dwilcox leaves
* dwilcox joins11:28
* dwilcox leaves11:29
* dwilcox joins
* dwilcox leaves11:30
* dwilcox joins11:31
* umgrosscol leaves
* dwilcox leaves11:32
<lsitu>escowles: It looks like we don’t have much options. Keeping those binary reference will lead to diff on disk when doing import and export again. Does it sounds good to just filtering out those binary references?
* dwilcox joins
<escowles>lsitu: yes, filtering them out sounds good — if the export is configured not to export binaries, then that seems like the right thing to do11:33
* dwilcox leaves
* dwilcox joins11:34
<awoods>escowles/lsitu: What is the use case for exporting without binaries?11:35
escowles/lsitu: I thought it was to reduce the size of the export
<escowles>awoods: yes, i think that it — allowing more frequent exports of metadata11:36
<awoods>escowles/lsitu: If we are removing any notion of those binaries on import... that seem problematic
<escowles>awoods: do you think the exported metadata should include links to the binaries, and we should filter them out on import?
that's the other option we talked about
* dwilcox leaves11:37
<awoods>escowles/lsitu: Let's approach it from the desired result of a round trip without binaries
<escowles>creating a placeholder seemed good at first, but then the checksum wouldn't match, which also seems like a bad result
<awoods>escowles/lsitu: I think we need a use case11:38
* dwilcox joins
<escowles>i think if you did a roundtrip without binaries, you would need to re-add the binaries afterwards, so filtering them out seems like a good option
* dwilcox leaves
<escowles>one i have is: i want to migrate from internal content to external content: export without binaries, import that into a new repo, and then add external content links to the binaries hosted elsewhere11:39
<awoods>escowles/lsitu: That is helpful
<escowles>another one might be: i want to export metadata frequently, and restore it, but leave binaries alone
* dwilcox joins11:40
* dwilcox leaves
<awoods>escowles/lsitu: it seems that filtering out the binary links would not work well for the second use case11:41
escowles/lsitu: although the filtering approach would be very good for the first use case
* dwilcox joins
* dwilcox leaves11:42
<escowles>awoods: the second case seems more like the syncing scenarios we haven't tackled yet
* dwilcox joins11:43
* dwilcox leaves
<awoods>escowles/lsitu: I am ok with documenting the first use case and targeting the filtering impl towards supporting that.
escowles/lsitu: It would probably be good to also document the second use case... and put it in the "not yet" category11:44
* dwilcox joins
<escowles>awoods: that sounds good to me — i think implementing something that works is better than the current broken behavior — we can get feedback and refine from there
<awoods>escowles/lsitu: +1
<escowles>i'm going to get some lunch right now — but i can write up those use cases after that
<awoods>escowles/lsitu: https://wiki.duraspace.org/display/FF/Design+-+Import+-+Export#Design-Import-Export-Usecases11:45
* dwilcox leaves
* dwilcox joins11:46
<lsitu>awoods/escowles: I’ll start from that approach to filter out the binary reference for now then. Thanks.
* dwilcox leaves11:47
<awoods>lsitu: +1
* dwilcox joins
* dwilcox leaves11:48
* dwilcox joins11:49
* dwilcox leaves
* dwilcox joins11:51
* dwilcox leaves11:52
* dwilcox joins11:53
* dwilcox leaves
* dwilcox joins11:54
* dwilcox leaves11:55
* coblej leaves
* dwilcox joins11:56
* dwilcox leaves11:57
* dwilcox joins
* dwilcox leaves11:58
* dwilcox joins11:59
* dwilcox leaves
* dwilcox joins12:00
* dwilcox leaves12:01
* dwilcox joins12:02
* dwilcox leaves
* dwilcox joins12:03
* dwilcox leaves12:04
* dwilcox joins12:05
* benpennell joins
* dwilcox leaves
* dwilcox joins12:06
* dwilcox leaves12:07
* dwilcox joins
* dwilcox leaves12:08
* dwilcox joins12:09
* dwilcox leaves
<westgard>need to leave for a meeting will be back online in the later pm
* westgard leaves
* dwilcox joins12:11
* dwilcox leaves
* dwilcox joins12:12
* dwilcox leaves12:13
* dwilcox joins12:14
* dwilcox leaves
* dwilcox joins12:15
* dwilcox leaves12:16
* dwilcox joins
* dwilcox leaves12:17
* dwilcox joins12:18
* dwilcox leaves
* dwilcox joins12:19
* dwilcox leaves12:20
* dwilcox joins
* dwilcox leaves12:21
<youn>Can I use fcrepo-webapp-4.7.2-jetty-console.jar to start a 2nd repository to import into? I tried using different port numbers, but that did not seem to work.
* dwilcox joins12:22
* dwilcox leaves12:23
* dwilcox joins
* dwilcox leaves12:24
* dwilcox joins
<mikeAtUVa>youn: do you need them both open at once, or just want to have two separate instance?
youn: running at once, that is.12:25
<youn>I just want two separate instances; they don't need to be running at once.
* dwilcox leaves12:26
<mikeAtUVa>youn: in that case recommend stopping the first one, moving the data directory somewhere else and starting it again. You'll find it empty upon the second start because fcrepo4-data contains all the content in the repository
youn: basically that fcrepo4-data directory for the one-click-run *is* your repository... delete it to start over fresh, copy it to create a backup, etc.
* dwilcox joins
* dwilcox leaves12:28
* dhlamb leaves
* dwilcox joins
* dwilcox leaves12:29
* dwilcox joins12:30
* dwilcox leaves
<youn>mikeAtUVa: Thanks. I cleared my browser cache and the problem went away.12:31
* github-ff joins
[fcrepo-specification] awoods pushed 1 new commit to master: https://git.io/v9dPn
fcrepo-specification/master 790c6c1 Benjamin Armintor: update reference to RFC 2616 to RFC 7231 for HTTP 1.1 (#117)...
* github-ff leaves
* dwilcox joins
* dwilcox leaves12:32
* dwilcox joins
* dwilcox leaves12:33
* dwilcox joins12:34
* dwilcox leaves
<youn>mikeAtUVa: I also had to rename the directory as you said.
* dwilcox joins12:35
* dwilcox leaves
* dwilcox joins12:36
* dwilcox leaves12:37
* dwilcox joins12:38
* dwilcox leaves
<escowles>awoods/lsitu: i added a couple lines to https://wiki.duraspace.org/display/FF/Design+-+Import+-+Export#Design-Import-Export-Usecases in the "use cases" and the "use cases yet to be rolled into requirements" sections
<awoods>thanks, escowles.12:39
* dwilcox joins
* dwilcox leaves
* dwilcox joins12:40
<awoods>youn: here are some specific steps for running two one-clicks at the same time... if desired: https://wiki.duraspace.org/display/FF/Sprint+3+Feedback+-+A+Woods#Sprint3Feedback-AWoods-Initialization
* dwilcox leaves12:41
* lsitu leaves
* lsitu joins12:42
* dwilcox joins
* dwilcox leaves
* dwilcox joins12:43
* dwilcox leaves12:44
<youn>thanks, awoods.
* coblej joins
* dwilcox joins12:45
* dwilcox leaves12:46
* dwilcox joins
* dwilcox leaves12:47
* dwilcox joins12:48
* dwilcox leaves
* dwilcox joins12:49
* dwilcox leaves12:50
* dwilcox joins
* dwilcox leaves12:51
* dwilcox joins12:52
* dwilcox leaves
* kestlund joins13:01
<coblej>When I run Fedora 4.7.2 via one-click run, what database is it using?13:07
<escowles>coblej: an embedded h2
<coblej>escowles: is there a way to clear it out?
e.g., if I want to set my repo back to ground zero13:08
<escowles>the easiest way is to remove the data directory
* dhlamb joins
<awoods>all: I am going to be away for the remainder of the day.
* awoods leaves
<coblej>escowles: I had done that but was still seeing nodes under rest/ ... turned out it was a browser cache issue ... thanks13:09
* coblej leaves13:28
* coblej joins13:44
* kestlund leaves13:56
<youn>I am trying to use pyenv with the local environment set to 3.5.2 but am getting: File "verify.py", line 21, in <module> from urllib.parse import urlparse, quote ImportError: No module named parse. I tried pip install urllib3 but am still getting the error.14:00
<coblej>Trying to load dataset from fcrepo-sample-dataset ... started up clean one-click Fedora 4.7.2 ... ran "mvn -Dfcrepo.url=http://localhost:8080/rest/ exec:java" as instructed ... no datasets loaded ... errors are "(FedoraInvalidNamespaceExceptionMapper) NamespaceExceptionMapper caught an exception: Prefix fcr has not been registered"14:19
* yamil leaves
<coblej>I was able to load a couple of the additional datasets (LUBM and plant patents) using the instructions for them14:20
just not the "main" ones (cityOfHeavenlyFire, cress, etc.)14:21
<youn>Can one use the one-click option with repository configuration options https://wiki.duraspace.org/display/FEDORA4x/Configuration+Options+Inventory#ConfigurationOptionsInventory-RepositoryConfigOptions? I am trying to look into https://jira.duraspace.org/browse/FCREPO-2438.14:22
* yamil joins14:29
* github-ff joins14:39
[fcrepo-import-export] escowles created delete-tombstones (+1 new commit): https://git.io/v9Feb
fcrepo-import-export/delete-tombstones 760e259 Esmé Cowles: Adding option to allow removing tombstones when re-importing over deleted resources
* github-ff leaves
* github-ff joins14:42
[fcrepo-import-export] escowles opened pull request #76: Adding option to allow removing tombstones when re-importing over deleted resources (master...delete-tombstones) https://git.io/v9FvG
* github-ff leaves
* bridgetalmas joins14:49
i have a question about importing binary resources. it seems the import tool expects any non-rdf resources to have a .binary extension. Is this a Fedora restriction? Is it something that is configurable?
<escowles>it's an import-export client thing - we decided to just have all binaries end with .binary and all external content links end with .external14:50
* benpennell leaves14:51
<escowles>we didn't really have any use cases for customizable extensions, but you can add one to the import-export design wiki page14:52
<bridgetalmas>ok thanks.14:53
* youn leaves14:55
* kestlund joins15:02
* peichman leaves15:03
* dwilcox joins15:06
* peichman joins
* coblej leaves15:11
* coblej joins
* dhlamb leaves15:12
* benpennell joins15:15
* coblej leaves15:16
* coblej joins15:19
* bridgetalmas leaves15:30
* benpennell leaves15:31
* dwilcox leaves15:33
* coblej leaves15:34
* umgrosscol joins15:39
* westgard joins15:56
* coblej joins16:00
* benpennell joins16:02
* umgrosscol leaves16:24
* peichman leaves16:30
* peichman joins16:32