Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* thomz joins02:44
* willtp87 joins06:30
* youn joins08:38
* coblej joins08:45
* informatician joins08:46
* coblej leaves08:47
* dhlamb joins08:48
* awead joins08:50
[Import/Export Standup]08:51
Finished Friday:
Still working
Working on today:
Reviewing https://jira.duraspace.org/browse/FCREPO-2225
Traveling to Penn State today, will be offline most of the day
* osmandin joins08:57
* coblej joins08:59
* awead leaves09:06
* coblej leaves09:27
* acoburn joins
* coblej joins09:28
* coblej leaves09:32
* github-ff joins09:33
[fcrepo-camel] birkland closed pull request #139: Add a method that can be used to extract multi-valued property values (master...fcrepo-2345) https://git.io/v1o7g
* github-ff leaves
* travis-ci joins09:38
fcrepo4-exts/fcrepo-camel#332 (master - a931692 : birkland): The build passed.
Change view : https://github.com/fcrepo4-exts/fcrepo-camel/compare/9f1272e0b7d5...a931692abead
Build details : https://travis-ci.org/fcrepo4-exts/fcrepo-camel/builds/183264034
* travis-ci leaves
* coblej joins
* github-ff joins09:40
[fcrepo-camel-toolbox] acoburn opened pull request #121: Make use of the property placeholder tokenize method in fcrepo-camel (master...fcrepo-2345) https://git.io/v16ec
* github-ff leaves
<escowles>[Import/Export Standup]09:44
friday: fixed bug where the importer was trying to parse binaries as RDF
* added support for custom membership predicates and sending checksums with binary uploads
* reviewed and merged PRs
today: starting on BagIt impl
* standing in for ruebot, so ping me if you need help/review/etc.
blockers: none
* awead joins09:49
* justinsimpson joins09:56
[Import/Export Standup]09:58
nothing - was sick on Friday, just getting back now
Working on today:
Updating docs for https://jira.duraspace.org/browse/FCREPO-2226
catching up on progress since Thursday
* ajs6f joins09:59
* github-ff joins
[fcrepo-camel-toolbox] birkland closed pull request #121: Make use of the property placeholder tokenize method in fcrepo-camel (master...fcrepo-2345) https://git.io/v16ec
* github-ff leaves
* awead leaves10:00
* westgard joins
* dbernstein joins10:01
* peichman joins10:02
* bseeger joins10:03
* osmandin leaves10:08
<westgard>[Import/Export Standup]10:09
Finished Friday:
created issue FCREPO-2347 (verification tool needs to infer extension) and did a PR to fix it
tested FCREPO-2332 (verification tool should handle up to date config file)
tested PR 58 (use YAML config) for the import-export tool
Working on today:
handle '.external' resources and better error handling
not a blocker but may be good to discuss: the main resource class in the verification tool probably needs to be refactored
* coblej leaves10:10
* travis-ci joins10:12
fcrepo4-exts/fcrepo-camel-toolbox#313 (master - bd6c81f : birkland): The build passed.
Change view : https://github.com/fcrepo4-exts/fcrepo-camel-toolbox/compare/09000acc3407...bd6c81ff5ca7
Build details : https://travis-ci.org/fcrepo4-exts/fcrepo-camel-toolbox/builds/183271486
* travis-ci leaves
* informatician leaves10:13
* informatician joins
[Import/Export Standup]
Finished Friday:
- No items finished. Broke our staging server cluster (reverted to snapshot taken while troubleshooting verify.py).
- Recovered / restored staging servers back to proper health.
Working on Today:
- Continuing to troubleshoot verification tool against fcrepo 4.0.
- Testing verify.py against different versions of python (currently 2.6.6, will also try 2.7.x, 3.4)
- Will continue to setup debugging environment for testing verify.py against fcrepo 4.0.
* coblej joins10:16
<bseeger>Informatician: I've tried testing the python script against 2.71 in python. It didn't work because of the `from os import scandir, stat` requirement. Just an FYI10:19
* coblej leaves10:20
<bseeger>Informatician: I think the scandir is part of python 3+.
* coblej joins
<bseeger>westgard, informatician: what do we think of requiring the verify tool to be run in python 3? It might be less of a concern for places where one can just install python 3. I wonder if other institutions would have an issue with this.10:24
* peichman leaves
<westgard>bseeger, informatician: I tend to do everything in Python 3 so I don't have a good sense of whether users with particular OS limitations or prferences would find that onerous or not. That said, I think we can workaround the scandir issue.10:26
<bseeger>westgard, informatician: I was just thinking that. There's os.walk (which is what I used in the other version). Looks like there are other plugins. That being said, I'd prefer not to have too many modules that folks need to fetch.10:27
* peichman joins10:30
<westgard>bseeger, informatician: os.walk would work, or os.listdir both are part of the standard library so we'd just need to change the imports (no additional requirements)10:33
the thinking with scandir is that it is the most direct way to the list of files but we're only talking about a couple extra lines with either of the above alternatives10:34
* whikloj joins10:36
<bseeger>westgard - the impression I got was that scandir runs faster. But if it's a matter of accessibility or speed… well, I'm not sure. I guess in this case we should go with accessibility.
* github-ff joins10:39
[fcrepo-camel-toolbox] acoburn opened pull request #122: Update installation instructions (master...fcrepo-2346) https://git.io/v163V
* github-ff leaves
<westgard>bseeger, ecowles: I wanted to bring up one larger issue with verify.py. I am finding that with the additions we've been making to handle nobinaries and external resources, the cleanest way forward at this point (for the tickets I'm working on) would be to refactor the main resource class. On the one hand this seems like a disproportionate change for the outstanding tickets at this point in the sprint, but on10:40
the other hand the whole script is only 435 lines so we're not talking about a lot of code.
The issue is that Resource class is starting to get rather spaghettified in my opinion because it is handling so many different cases. It would be better to have a few subclasses of resource depending on whether they are in fedora or on disk.10:42
* acoburn leaves10:43
<escowles>westgard: that seems reasonable to me — i think it's better to address things like that as they come up10:45
<westgard>escowles: sorry mistyped your handle above. Do you have thoughts about this ^?
<escowles>westgard: i think it's fine to refactor when stuff starts getting unwieldy — even if it's not driven by the particular ticket you're working10:46
<bseeger>westgard, escowles: I'd be +1 for refactoring the file.
* youn leaves10:47
<westgard>OK, thanks. I already started thinking about how to do it in the context of the ticket to handle external binaries and I think I can get it done in the next couple days.
I wanted to make sure you both were on board with the scope of that particular PR potentially extending beyond the immediate task of dealing with externals10:48
<westgard>bseeger +1 on the question of accessibility trumping speed for this tool
* acoburn joins10:56
<informatician>bseeger: Excellent, thanks for the tip re: python. I'll see about switching up to python 3.x.10:57
<bseeger>informatician: Just a FYI — I am able to have python 2.71 and python3 on the same machine. My python defaults to 2.71 and I have to use python3 to run python 3. I think this might be standard, as it was how it installed on my machine (on mac)11:01
* youn joins
* github-ff joins11:15
[fcrepo-camel-toolbox] dannylamb closed pull request #122: Update installation instructions (master...fcrepo-2346) https://git.io/v163V
* github-ff leaves
<informatician>bseeger, westgard: I was just talking over the question of requiring python 3.x for the verification tool with some colleagues here. My thought is that python 2.6/2.7 are going to be the default installs for the majority of the hydra community for some time to come, however I also don't think the folks who would actually be running the exports woul11:24
d encounter much issue making a python 3.x environment available on their boxes. IMHO, just making it explicit that python version 3.x+ is required should do the trick.
From the earlier tip about encountering issues with python 2.x -->11:25
* travis-ci joins11:29
fcrepo4-exts/fcrepo-camel-toolbox#315 (master - 7f744c1 : dannylamb): The build passed.
Change view : https://github.com/fcrepo4-exts/fcrepo-camel-toolbox/compare/bd6c81ff5ca7...7f744c1a8183
Build details : https://travis-ci.org/fcrepo4-exts/fcrepo-camel-toolbox/builds/183295018
* travis-ci leaves
<westgard>informatician++ -- bseeger, I am OK requiring python3 as the minimum shippable product. Depending on how things play out this week, we can also keep an eye on the issue of python2 support. I suspect it may not be all that hard to implement.11:31
<bseeger>westgard, informatician — westgard, that sounds like a good approach. I do think it would be valueable to have it working in python 2, but see no immediate need for it11:34
* youn leaves11:43
* westgard1 joins11:44
* westgard leaves11:46
* ajs6f leaves11:49
* justinsimpson leaves
* coblej leaves11:50
* bseeger leaves11:51
* acoburn leaves12:13
* coblej joins12:19
* westgard1 leaves12:23
* coblej leaves12:24
* westgard joins
<dbernstein>[Import/Export Standup]12:25
Finished yesterday:
Implement binary redirects export approach
Pull request tweaked and accepted
Working on today:
I’m not sure: let’s talk about it.
<escowles>dbernstein: can you review and test https://github.com/fcrepo4-labs/fcrepo-import-export/pull/54 ?12:27
<dbernstein>sure - I was just going to ask what I can do.
<escowles>i'm working on an initial bagit export impl right now, and we should be able to parallelize adding different profiles once that's done12:28
<dbernstein>escowles: great.12:31
Hey - can anyone tell me how to checkout a pull request for testing?12:32
<escowles>dbernstein: there are instructions at the bottom — they boil down to creating a new branch and then pulling the related branch into it12:35
there should be a link called "command-line instructions" that will un-hide the instructions when you click on it
* coblej joins12:45
<dbernstein>oh I see - upload-fixity is the branch. Got it.12:46
* bseeger joins12:51
* github-ff joins13:00
[fcrepo-import-export] escowles created export-bags (+1 new commit): https://git.io/v16S4
fcrepo-import-export/export-bags 246c0e7 Esmé Cowles: Initial BagIt export implementation.
* github-ff leaves
* github-ff joins13:01
[fcrepo-import-export] escowles opened pull request #59: Initial BagIt export implementation. (master...export-bags) https://git.io/v16SK
* github-ff leaves
* acoburn joins13:02
* dhlamb leaves13:06
* dwilcox joins
* dhlamb joins13:08
* bseeger leaves13:11
* dhlamb leaves13:12
* dhlamb joins
* dwilcox leaves13:23
* coblej leaves13:26
* coblej joins13:30
<escowles>dbernstein: whikloj: i've added several new tickets to the BagIt epic: https://jira.duraspace.org/browse/FCREPO-222113:37
<whikloj>escowles++ # I'm trying out export-bags now
<escowles>i think FCREPO-2350, FCREPO-2351, FCREPO-2352, and FCREPO-2353 are ready and we should be able to split them up and work in parallel13:38
* coblej leaves13:39
<whikloj>escowles: do you use shasum? I tried to pass it the generated manifest-SHA1.txt and it doesn't like the format for some reason13:41
<escowles>whikloj: i saw the same thing — i expected it to work, but maybe the bag format is not compatible?13:42
i think i'll make a ticket to read the spec and see if it can be compatible, because that would obv. be great for validating quickly
<whikloj>escowles: hrm, ruebot_away told me about a python library you can use maybe I'll try that. Just wondering if it was me.13:43
* coblej joins13:45
* osmandin joins13:55
* osmandin leaves13:57
<whikloj>escowles: Ok, so we are missing the Payload-Oxum which the Python bagit needs to actually verify the data. That is an optional element, so we should probably make sure to add that.13:59
<escowles>whikloj: yep, that's my first priority in https://jira.duraspace.org/browse/FCREPO-2353
* coblej leaves14:02
* bseeger joins14:03
* acoburn leaves14:04
<whikloj>escowles: So the manifest file is not properly formatted, it is missing the * at the start of the file path. So perhaps we can fix that in their library14:05
<escowles>whikloj: the spec says the asterisk is optional: https://tools.ietf.org/html/draft-kunze-bagit-08#section-2.1.314:07
<whikloj>escowles: See you can also add a second space and it works on my Mac. But a single space doesn't work. Is there a standard for SHA-1 file format?14:08
<escowles>whikloj: i don't know — it would be nice if there was14:09
but the bagit spec says "One or more linear whitespace characters (spaces or tabs)" so if adding a second space makes it work, then I think putting in an issue and/or PR to the bagit library is a good idea14:10
<whikloj>escowles: agreed
* coblej joins14:13
* coblej_ joins
* github-ff joins
[fcrepo-import-export] whikloj closed pull request #59: Initial BagIt export implementation. (master...export-bags) https://git.io/v16SK
* github-ff leaves
* coblej leaves14:17
* coblej_ leaves14:20
* coblej joins
* dwilcox joins
* coblej_ joins14:23
* coblej leaves
* coblej_ leaves14:24
* coblej joins14:26
* acoburn joins14:38
* dwilcox leaves14:39
* coblej leaves14:40
* westgard leaves14:44
* coblej joins14:50
[Import/Export Standup]14:54
Completed Yaml config changeover for import/export and verify tooling14:56
Going to try FCREPO-2351 (ITs for BagIt)14:57
* github-ff joins
[fcrepo-camel] bseeger opened pull request #140: Adds developer section to pom. (master...dev-update) https://git.io/v1iqe
* github-ff leaves
* peichman1 joins15:01
* github-ff joins15:02
[fcrepo-import-export] escowles created export-bags-techmd (+1 new commit): https://git.io/v1imF
fcrepo-import-export/export-bags-techmd a142093 Esmé Cowles: Adding basic technical metadata to exported Bags
* github-ff leaves
* peichman leaves15:03
* github-ff joins
[fcrepo-import-export] escowles opened pull request #60: Adding basic technical metadata to exported Bags (master...export-bags-techmd) https://git.io/v1iYv
* github-ff leaves
* peichman joins15:14
* peichman1 leaves
<whikloj>escowles: It seems that the BagIt community would suggest we alter our path, do you see a problem with sticking the * in front?
<escowles>alter our path by making the path "*data/foo/bar/file.txt" ?15:15
imho, if they're open to a PR, we should open a PR to put the * in front — the spec allows it, and it'll trigger the correct behavior in shasum/md5sum15:16
* github-ff joins15:21
[fcrepo-camel-toolbox] bseeger opened pull request #123: Adds developer information (master...dev-update) https://git.io/v1iG9
* github-ff leaves
* dbernstein leaves
* github-ff joins15:25
[fcrepo-camel] dannylamb closed pull request #140: Adds developer section to pom. (master...dev-update) https://git.io/v1iqe
* github-ff leaves
* travis-ci joins15:26
fcrepo4-exts/fcrepo-camel#335 (master - ede2d6e : B. Seeger): The build has errored.
Change view : https://github.com/fcrepo4-exts/fcrepo-camel/compare/a931692abead...ede2d6ef9197
Build details : https://travis-ci.org/fcrepo4-exts/fcrepo-camel/builds/183369487
* travis-ci leaves
* dwilcox joins15:41
* acoburn leaves15:51
* acoburn joins15:53
* acoburn leaves16:02
* peichman leaves16:20
* peichman joins16:22
* github-ff joins16:27
[fcrepo-import-export] whikloj created missing-output (+1 new commit): https://git.io/v1i04
fcrepo-import-export/missing-output bacefd0 Jared Whiklo: Adding a couple of missing fields to the config output
* github-ff leaves
* github-ff joins
[fcrepo-import-export] whikloj opened pull request #61: Adding a couple of missing fields to the config output (master...missing-output) https://git.io/v1i0a
* github-ff leaves
* github-ff joins16:55
[fcrepo-import-export] escowles closed pull request #61: Adding a couple of missing fields to the config output (master...missing-output) https://git.io/v1i0a
* github-ff leaves
* coblej leaves17:01
* dwilcox leaves17:04
* peichman leaves17:25
* bseeger leaves
* whikloj leaves18:00
* dhlamb leaves18:45
* informatician leaves19:05
* informatician joins19:20
* acoburn joins20:49
* acoburn leaves21:13
* github-ff joins22:26
[fcrepo-import-export] whikloj created FCREPO-2351 (+1 new commit): https://git.io/v1Pko
fcrepo-import-export/FCREPO-2351 b9c1855 Jared Whiklo: New BagIt test
* github-ff leaves
* github-ff joins22:29
[fcrepo-import-export] whikloj opened pull request #62: BagIt IT (master...FCREPO-2351) https://git.io/v1Pk9
* github-ff leaves

Generated by Sualtam