Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* manez joins00:54
* benpennell leaves00:55
* manez leaves00:59
* f4jenkins leaves02:17
* f4jenkins joins02:18
* manez joins02:56
* manez leaves03:00
* manez joins03:57
* manez leaves04:01
* manez joins04:58
* manez leaves05:02
* dhlamb joins05:19
* dhlamb leaves05:34
* manez joins05:58
* manez leaves06:03
* manez joins06:59
* manez leaves07:04
* manez joins07:19
* manez leaves07:31
* manez joins07:50
* dwilcox joins07:53
* youn joins08:04
* awoods joins08:12
* dhlamb joins08:15
* manez leaves08:21
* coblej joins08:33
[Import/Export Standup]08:36
Finished yesterday:
Working on today:
* manez joins
<coblej> Testing sprint changes using interim code releases
Document import structural expectations — https://jira.duraspace.org/browse/FCREPO-2455
* manez leaves08:41
* dwilcox leaves08:45
* manez joins08:52
* benpennell joins08:54
* benpennell leaves08:56
* benpennell joins
* manez leaves08:59
* mikeAtUVa joins09:00
* umgrosscol joins09:17
<awoods>Import/Export Standup]09:23
Finished yesterday:
- Reviewed tickets
- Created testing recipes (please everyone do at least one unique test): https://wiki.duraspace.org/display/FF/Import+-+Export+Testing+Recipes
- Uploaded recent versions of import/export utility and Fedora-master: https://wiki.duraspace.org/display/FF/2017-05+Import+-+Export+Sprint+04+Meetings#id-2017-05Import-ExportSprint04Meetings-Resources
- Created ticket for testing import into Cavendish: https://jira.duraspace.org/browse/FCREPO-2476
Working on today:
- Reviewing tickets
- Waiting on resolution to: new bug "import-export does not respect containment predicates when pointed at rest/" (https://jira.duraspace.org/browse/FCREPO-2474)
coblej: If you have time, I am asking everyone on the sprint to perform (and document on the wiki) at least one unique test as detailed here: https://wiki.duraspace.org/display/FF/Import+-+Export+Testing+Recipes09:25
<coblej>awoods: ok ... should be able to get to that shortly09:26
* dhlamb leaves
* yamil joins
* dwilcox joins09:27
* dwilcox leaves
* dwilcox joins09:28
* dwilcox leaves09:29
* dwilcox joins
* dwilcox_ joins09:32
* dwilcox leaves09:34
<mikeAtUVa>[import/export sprint standup]09:36
- https://github.com/fcrepo4-labs/fcrepo-import-export/pull/88 updated importer to support lossless roundtripping of RDF
- https://jira.duraspace.org/browse/FCREPO-2464 (document "relaxed" triples mode)
- I'll take a look at https://github.com/fcrepo4-labs/fcrepo-import-export/pull/85 if it's not already reviewed/merged
- Look into aligning versioning with spec, providing feedback where necessary09:37
- none
<awoods>mikeAtUVa: If you have time, I am asking everyone on the sprint to perform (and document on the wiki) at least one unique test as detailed here: https://wiki.duraspace.org/display/FF/Import+-+Export+Testing+Recipes
<youn>[Import/Export Standup]09:44
Finished yesterday:09:45
- draft proposal for videos and ***WELCOME YOUR FEEDBACK*** https://docs.google.com/document/d/1CzvJlZ3m3eEhHgVJJY0hdGerPMxRIRaK9X05WhCLKpo/edit?usp=sharing
Working on today:
- test bag import export and version import export
* lsitu joins09:46
<youn>- test predicates and inbound with lubm
- https://jira.duraspace.org/browse/FCREPO-2473 (verification tool should handle import-export v1.1 config files)
<lsitu>[Import/Export Standup]
Finished yesterday:
Improved and rebased the PR for Import of Bags should verify binary digest: https://jira.duraspace.org/browse/FCREPO-2418
Working on today:
Not sure: can help on anything else.
- none
<escowles>[Import/Export Standup]09:47
* Exporting members based on inbound links https://jira.duraspace.org/browse/FCREPO-2453
* Fix Bag import using config file https://jira.duraspace.org/browse/FCREPO-2462
* Trailing slash issues https://jira.duraspace.org/browse/FCREPO-2467
* Containment predicates don't work when pointed at /rest https://jira.duraspace.org/browse/FCREPO-2474
* Squashing bugs from testing
* dwilcox_ leaves
<escowles>* None
<benpennell>[Import/Export Standup]09:50
Finished yesterday:
* A bunch of local stuff
* Some decisions about requirements for import of versions related to timestamps
Working on today:
* Implementation of version import https://jira.duraspace.org/browse/FCREPO-2459
* none
<awoods>youn: can you please link your video proposal to the related ticket? https://jira.duraspace.org/browse/FCREPO-229909:52
* github-ff joins
[fcrepo-import-export] coblej opened pull request #89: Add file describing import-export format. Addresses one portion of F… (master...fcrepo-2455-import-documentation) https://git.io/vHmxt
* github-ff leaves
<awoods>youn/lsitu/escowles/benpennell: If you have time, I am asking everyone on the sprint to perform (and document on the wiki) at least one unique test as detailed here: https://wiki.duraspace.org/display/FF/Import+-+Export+Testing+Recipes09:53
* peichman joins09:55
* dwilcox joins
<youn>awoods: done and will do.
<coblej>awoods: Not sure of the procedure to use in this case ... submitted PR that addresses a portion of https://jira.duraspace.org/browse/FCREPO-2455 but does not satisfy all that is asked for in the issue (this may, however, be about all that I can do on this in the current sprint) ... do I mark the Jira issue as Start Review?
<awoods>coblej: yes, please put the ticket into review.09:57
<coblej>awoods: thanks
awoods: I put it in review with a resolution of "Incomplete"09:58
<awoods>coblej: I can work with that ;)
<lsitu>awoods: Sure. I’ll perform the test.10:00
* github-ff joins10:04
[fcrepo-import-export] escowles pushed 1 new commit to trailing-slash: https://git.io/vHmpj
fcrepo-import-export/trailing-slash b8389fd Esmé Cowles: Log a warning when one of the source/destination URIs ends with a slash and the other doesn't
* github-ff leaves
* github-ff joins10:07
[fcrepo-import-export] escowles opened pull request #90: Improve mapping when source and/or destination URIs end with slashes (master...trailing-slash) https://git.io/vHmh9
* github-ff leaves
* dbernstein joins10:09
[Import/Export Standup]
Finished yesterday:
Working on today:
* finishing up https://jira.duraspace.org/browse/FCREPO-2408
(make verification tool work with BagIt Bags)this morning
* https://jira.duraspace.org/browse/FCREPO-2475
(verifcation tool should log to timestamped log files)
* coblej leaves10:12
<awoods>dbernstein: If you have time, I am asking everyone on the sprint to perform (and document on the wiki) at least one unique test as detailed here: https://wiki.duraspace.org/display/FF/Import+-+Export+Testing+Recipes10:13
<dbernstein>oh - sure. I’ll add mine in.
* bseeger joins10:21
* coblej joins10:24
* whikloj joins10:29
<youn>I restored LUBM_02: https://github.com/fcrepo4-labs/fcrepo-sample-dataset/tree/master/lubm and tried to export using: https://gist.github.com/younnoh/c7980e78a83bfd32c60f317361b0cd7d but none of the resources under rest were exported.10:40
most of the resources are under http://localhost:8080/rest/Department0.University0.edu10:42
I used the one-click for 4.7.3-RC and the import export snapshot.10:44
<awoods>youn: yes, that is due to a new bug... the fix to which is being reviewed now: https://jira.duraspace.org/browse/FCREPO-247410:45
* github-ff joins10:49
[fcrepo-import-export] awoods closed pull request #90: Improve mapping when source and/or destination URIs end with slashes (master...trailing-slash) https://git.io/vHmh9
* github-ff leaves
<awoods>youn: try this one: https://s3.amazonaws.com/f4-artifacts/fcrepo-import-export-0.1.1-SNAPSHOT-d4780ca2.jar10:58
* github-ff joins10:59
[fcrepo-import-export] escowles created serialized-config (+1 new commit): https://git.io/vHYIl
fcrepo-import-export/serialized-config 339095e Esmé Cowles: Adding auditLog to serialized config, adding tests for serialized config
* github-ff leaves
* github-ff joins11:00
[fcrepo-import-export] escowles opened pull request #91: Adding auditLog to serialized config, adding tests for serialized config (master...serialized-config) https://git.io/vHYIV
* github-ff leaves
* mcritchlow joins11:05
* mcritchlow_ joins
* mcritchlow_ leaves11:06
<coblej>awoods: When updating a PR in response to review comments, it is generally preferred to squash commits or to leave the changes based on the review comments as a separate commit?11:24
awoods: The case in hand is updating the documentation PR based on your review comments.11:25
* bseeger leaves11:26
<awoods>coblej: great question. Please add your incremental updates based on code review comments as new commits to your branch... that way it is easier for the reviewer to see exactly what changed. When the iterative review process is over, the person who does the merge will then squash.
<coblej>awoods: will do
<awoods>thanks, coblej
team: I am going into meetings from now until 3:30pm ET... wish me luck.11:27
* bseeger joins11:44
* coblej leaves11:50
* bseeger leaves12:20
<youn>I ran export on LUBM_02 with the latest import export snapshot and this config https://gist.github.com/younnoh/0fa81303328ee7d877948672a792736e and got an error when trying to retrieve external content.12:21
ERROR 12:14:32.124 (ImportExportDriver) Error performing import/export: resource not found at http://localhost:8080/rest/Department0.University0.edu/FullProfessor0/Publication15/Proxy12:22
http://localhost:8080/rest/Department0.University0.edu/FullProfessor0/Publication15/Proxy is the page http://www.example.com/file (I am not sure I am linking to external content correctly.)
<mikeAtUVa>Can you get the external content if you navigate to it in your running repository?12:23
<mikeAtUVa>... I'm getting set up to try it as well.12:38
* lsitu leaves12:43
* manez joins
* lsitu joins
<youn>mikaeAtUVa: Thanks. This is how the resource was created: curl -u fedoraAdmin:secret3 -i -X PUT -H "Content-Type: message/external-body; access-type=URL; URL=\"http://www.example.com/file\"" "http://localhost:8080/fcrepo/rest/Department0.University0.edu/FullProfessor0/Publication15/Proxy" for the backup.12:45
<mikeAtUVa>youn: that when you go here (http://localhost:8080/rest/Department0.University0.edu/FullProfessor0/Publication15/Proxy) in a browser it displays the right page suggests the export code is in error, but I'll dig a little more12:47
youn: curl shows a redirect, so I'm pretty sure the external link is correct "curl -vv "http://localhost:8080/rest/Department0.University0.edu/FullProfessor0/Publication15/Proxy""12:48
* coblej joins12:51
* coblej leaves12:55
<mikeAtUVa>youn: aha... it seems http://www.example.com/file,while returning valid HTML reports the status code 404 (not found).
<youn>okay, thanks for investigating!!!12:57
<mikeAtUVa>youn: so this isn't a problem with the exporter, but instead we now know that example.com urls are suitable for this kind of testing.
youn: you're welcome. Thanks for testing!12:58
* umgrosscol leaves12:59
* coblej joins13:04
* bseeger joins13:06
* dhlamb joins
* umgrosscol joins13:13
<youn>curl -I https://wiki.duraspace.org/display/FF/Design+-+Import+-+Export returns 200. Would it be okay to use? I want to regenerate the data set so others can use it for testing in future sprints.
* rosiel joins13:17
<mikeAtUVa>youn: yeah though because it's a dynamic page and confluence is a behemoth, the cost of hitting it a bunch might be non-zero how about https://www.google.com, they're optimized for high traffic, and https is probably a good thing to test too.13:31
<youn>mikeAtUVa: great, thanks!!13:45
<benpennell>mikeAtUVa: some parts of trying to shoehorn versions into the import script are boggling my mind a bit (some assumptions don't line up), but one thing I'm more worried about is ldp:IndirectContainers…13:56
mikeAtUVa: namely it seems that when you make a GET request to a resource that has autogenerated relationships (lets say pcdm:hasMember) from an indirectcontainer/proxies you get back those pcdm:hasMembers. But when you make a GET request to the versioned resource it does not return the pcdm:hasMembers13:57
it looks like DirectContainers do come back with the autogenerated relations (pcdm:hasFile for example), so perhaps its related to the ldp:proxyFor referencing something outside of the versioned path.14:01
<awoods>benpennell: I suspect you have unearthed a bug... or at least a shortcoming in Fedora that has not been previously explored.14:07
<benpennell>awoods: should i put together a bug ticket?14:08
<awoods>youn: I also tend to use the following as external content: http://www.drodd.com/html7/${x}.html (where ${x} can be any number between 1 and 72 and any letter in the alphabet)14:09
* mikeAtUVa is not at all surprised... and still not convinced of the desirability of those ldp construct.
<awoods>benpennell: yes please... with a detailed set of steps for reproducing the behavior14:10
<benpennell>mikeAtUVa: another more general mismatch with the importer is that it imports all membership objects at the end, which doesn't work so well when replaying snapshots
<mikeAtUVa>benpennell: in order to build the snapshots you have to import the whole subgraph (or at least the part that's represented as children in modeshape) from the version and then make the snapshot.14:11
<benpennell>mikeAtUVa: yeah, which is fairly out of step with the existing workflow. so I may have to replicate or refactor a bunch of stuff14:12
i'm kind of wishing it operated in a more object oriented way so the pieces of a resource were processed together since i need to compute all the pieces of a version and then execute them one after the other, versus a file by file crawl14:14
<mikeAtUVa>benpennell: is it time to introduce transactions?14:16
<awoods>benpennell/mikeAtUVa: it is completely out of scope in this sprint, but the idea has been floated of using a triplestore in conjunction with import/export as an interim step
<benpennell>mikeAtUVa: the fedora variety or do you mean as a general approach in the import tool?14:17
<mikeAtUVa>benpennell: I meant the fedora variety, though I don't think it would actually help in this case, except possibly with performance.14:18
* manez leaves
<mikeAtUVa>benpennell: I usually implement in a copy/paste fashion, make sure it actually works, then refactor to remove the redundancy when faced with these. But that might be a reason it takes me so long sometimes to get small (but complex) changes done.14:19
<benpennell>mikeAtUVa: i'm a bit worried about refactoring something that a bunch of other people are working on at the same time too14:21
awoods: having a triplestore would be handy for querying against, but i'm not sure it'd cure what's currently ailing me14:22
* manez joins14:23
<awoods>benpennell: I am guessing I do not know you well enough to comment.
<benpennell>awoods: just a moderate case of refactoritis and mild indigestion14:27
* rosiel leaves14:28
* manez leaves
<awoods>benpennell :)14:29
<lsitu>awoods: I ran export on plants.tar.gz and got an error with binary exclude from the dataset. Three records in /audit have reference resource <http://localhost:8080/fcrepo/rest/initialize> but Fedora response with 404 (ResourceNotFoundRuntimeException) for head. I don’t see the /initialize resource are created in Fedora.14:51
In this case those parent resources that have references to a missing resource like <http://localhost:8080/fcrepo/rest/initialize> won’t be exported. Is it the expected behavior?14:56
<benpennell>awoods: bug ticket for the indirect container/version issue https://jira.duraspace.org/browse/FCREPO-247915:01
<coblej>Just a reality check on the updated versions of the various resources jar files ... if I run the fcrepo-webapp-4.8.0-SNAPSHOT-jetty-console-a07a1a46.jar with the "-Dfcrepo.properties.management=relaxed" property and use the import-export-0.1.1-SNAPSHOT-d4780ca2.jar, should I expect to see the original fedora:created and fedora:lastModified date stamps on the reimported objects?15:13
is there something else I need to do to overwrite those server-managed date stamps?15:14
awoods: mikeAtUVa: ^15:15
* awoods on a call
<dbernstein>awoods: would you mind, when you get a minute to review and merge my housekeeping branch.15:16
* manez joins15:17
* manez leaves
<youn>awoods: Thanks for the link to use for external content. I made a new backup, but it is too big to upload to my fork of the repository on GitHub, so I put it here https://yale.box.com/s/0rkv4q1akg7g1w3oq4zhnrkafeppmplg . The other two lubm backups should be deleted. Thank you!15:28
<dbernstein>youn: I was going to start working on https://jira.duraspace.org/browse/FCREPO-2456 (support for versions) based on the changes benpennell is making in https://github.com/fcrepo4-labs/fcrepo-import-export/pull/85 (fcrepo-2458).15:30
Just calling it out, since I see you’re working on https://jira.duraspace.org/browse/FCREPO-2473?src=confmacro15:31
<youn>dbernstein: That sounds great. I'm only hoping to test versions not to do any work on the verification tool related to versions.
I'll start on 2473 tomorrow.15:32
<dbernstein>okay - cool just wanted to mention it since there’s a new “versions” parameter in the import export file coming down the pike - just wanted to make sure you didn’t spend any cycles on that.15:33
<benpennell>dbernstein: just to warn you i am fixing some merge conflicts on my branch at the moment
<youn>thanks for the heads up
<mikeAtUVa>coblej: the import tool won't support posting those modification dates and such until https://github.com/fcrepo4-labs/fcrepo-import-export/pull/88 is merged
* coblej leaves15:36
* manez joins15:38
<benpennell>dbernstein mikeAtUVa: okay merge conflicts are resolved again, for the moment15:44
* dhlamb leaves15:50
* westgard joins15:51
* coblej joins15:52
mikeAtUVa: Thanks!
<awoods>benpennell/mikeAtUVa/lsitu: I have a few minutes right now, do you have a preference for a ticket/PR to review/merge first?15:55
* manez leaves15:56
<benpennell>awoods: i don't really know what you're picking between, but the version export one is at least merge conflict free for the moment :)15:58
<lsitu>awoods: I just opened two tickets for the bugs. Should I use the master branch to verify import/export, or the one from https://github.com/fcrepo4-labs/fcrepo-import-export-verify/releases/tag/0.0.3?15:59
* youn leaves16:01
<awoods>westgard/dbernstein: do you have a recommendation for lsitu regarding what version of the verification tool he should use?16:02
<westgard>awoods, lsitu: I think master is fine, but dbernstein may have a different suggestion.16:03
<dbernstein>yes - I agree.
<lsitu>westgard/dbernstein: I’ll use the master then. Thanks.16:05
<coblej>escowles: I have a question about the inbound references functionality added to the import-export tool, if you have a moment16:08
<escowles>coblej: sure16:09
<coblej>escowles: I was expecting to be able to use it to export a resource (say, a collection resource) and all the other resources that have a particular relationship to that object (e.g., isMemberOfCollection) ... is that what this functionality is?16:11
<escowles>yes, that's how it's supposed to work
there's an integration test that creates a collection and two objects that link to it with a custom predicate, and exports the collection with inbound links: https://github.com/fcrepo4-labs/fcrepo-import-export/pull/84/files#diff-2085f9c938964a6c935f6979bfc109dbR23616:13
<coblej>escowles: thanks ... I'll look at that ... the options I gave it just exported the collection resource and no other objects16:15
<dbernstein>westgard: I just merged your pull request fyi. Also just pushed up another: https://github.com/fcrepo4-labs/fcrepo-import-export-verify/pull/35 if you have a moment to take a look at that.
<westgard>dbernstein: thx looking at PR35 now. It'll take a few minutes to test with bags.16:16
* manez joins16:18
* bseeger leaves16:19
* rosiel joins16:20
<coblej>escowles: got it ... I blundered in setting the predicate option properly on my first attempt
<escowles>coblej: great — so it's working for you now?16:21
<coblej>escowles: looks like it ... about to run verify tool
escowles: another blunder ... trying again :-)16:23
<escowles>nothing like fiddling with predicates to make us all into butterfingers :-)16:24
<awoods>coblej: please update the wiki when you have successfully run through a full round-trip: https://wiki.duraspace.org/display/FF/Import+-+Export+Testing+Recipes16:25
benpennell: I am seeing merge conflict with PR#85 and master16:27
<coblej>awoods: will do ... though I don't think I'll be able to do a successful roundtrip yet since the import operation doesn't yet support overwriting server-managed triples (if I understood mikeAtUVa correctly)
<awoods>coblej: yes, you understand him correctly.
<coblej>westgard: looks like the verify tool does not handle inbound references correctly? Is that covered under https://jira.duraspace.org/browse/FCREPO-2473 (import-export v1.1 config files) or should I put in a ticket about that specifically?16:32
* dwilcox leaves16:37
<coblej>awoods: I updated the fcrepo-import-export documentation PR based on your review comments and put the Jira ticket back in the Review state ... is that all I need to do to get it back on your radar?16:39
<westgard>coblej: thanks. It is part of what was intended to be captured by the https://jira.duraspace.org/browse/FCREPO-2473 ticket, but if there's a specific bug that you've observed it would be useful to have a more specific ticket. I have to admit that I haven't even thought about what the implications of inbound refs would be.16:45
<coblej>westgard: what I saw when I ran the verify tool against an export that used the inbound option was a mismatch on the RDF for the resource with the inbound references. I assume it's because the exported resource .ttl contains the inbound properties but, of course, the fcrepo object doesn't16:47
<awoods>coblej: yes, its on my radar16:49
<coblej>awoods: just wanted to be sure I had the correct procedure :-)
<awoods>coblej: ...my bottlenecky radar16:50
<coblej>awoods: wasn't trying to poke you ... as I said, just trying to make sure I was following the correct procedure16:51
* github-ff joins
[fcrepo-import-export] awoods pushed 1 new commit to master: https://git.io/vHOkI
fcrepo-import-export/master 5103ee1 Jim Coble: Add file describing import-export format. Addresses one portion of F… (#89)...
* github-ff leaves
<awoods>coblej: feel free to reopen that ticket with a comment that describes what else you have in mind for completing it. It seems pretty reasonable, where you finished.16:52
all: I will finish review of all "In Review" tickets tonight.16:53
<coblej>escowles: I assume I should be able to import an export that contains inbound references?16:54
<escowles>coblej: i hope so!
* github-ff joins16:56
[fcrepo-import-export] escowles created find-root-fallback (+1 new commit): https://git.io/vHOkd
fcrepo-import-export/find-root-fallback a8fa689 Esmé Cowles: Making findRepositoryRoot more robust
* github-ff leaves
* github-ff joins16:57
[fcrepo-import-export] escowles opened pull request #92: Making findRepositoryRoot more robust (master...find-root-fallback) https://git.io/vHOIf
* github-ff leaves
<dbernstein>Does anyone know how we can start getting notifications in irc from fcrepo-import-export-verify (as we are from the other projects)?
<coblej>escowles: I wiped and restarted fcrepo ... tried to import previous export using the same config file (but with mode changed to import) ... got ERROR 16:45:04.689 (Importer) Error while importing /Users/coblej/Projects/F4-Import-Export-Sprint/import-export/export-SNAPSHOT-dulhydra-inbound/rest/dev/bc/6b/8d/ff/bc6b8dff-245b-478d-92b2-dee042b54862.ttl (403): http://localhost:8080/rest/dev/b2/5f/c7/b8/b25fc7b8-7864-49a2-b4f2-a16:58
cfcdba32ec3 is not in the topic of this RDF, which is http://localhost:8080/rest/dev/bc/6b/8d/ff/bc6b8dff-245b-478d-92b2-dee042b54862 ... I guess I have something in the config wrong?
<escowles>coblej: maybe — but it's entirely possible there's some bug in the import code that's causing that16:59
i'm about to head out in a minute, but i'll take a look tomorrow morning to see if it works for me locally
<coblej>k ... let me know if you want me to open a Jira ticket ... or if you want to just look at it first17:00
<escowles>coblej: why don't you open a ticket — makes it easier to track it
<coblej>escowles: will do17:01
* rosiel leaves17:13
* coblej leaves
* coblej joins17:16
<lsitu>westgard/dbernstein: Following the README to setup and run “$ fcrepo-verify ./importexport.yml”, I got error “pkg_resources.DistributionNotFound: The 'scandir' distribution was not found and is required by fcrepo-verify” from the master branch.17:17
* coblej leaves
<lsitu>And I still got it after I installed it with “pip install scandir”. Do you know what’s wrong?
* yamil leaves
<dbernstein>try pip3 install -r requirements.txt .
we may need to update the README to show how to install the tool on the path.17:18
lsitu: let me know if that works.
<westgard>lsitu: yes what dberstein said -- pip will only install libraries that work with the default python, so you must specify pip317:19
python 2 is still the default most everywhere
<lsitu>: westgard/dbernstein: I just tried “pip3 install -r requirements.txt”, but I still got the error.17:20
<dbernstein>did it install successfully?
<westgard>try manually pip3 install scandir
that is: "pip3 install scandir"17:21
* umgrosscol leaves
<lsitu>Yes. It’s installed successfully: Installing collected packages: pyflakes, pycodestyle, mccabe, flake817:22
Successfully installed flake8-3.3.0 mccabe-0.6.1 pycodestyle-2.3.1 pyflakes-1.5.0
<dbernstein>also you might need to preface the pip3 with sudo -H as in “sudo -H pip3 install -r requirements.txt .”17:23
also notice there is a “.” at the end of the command.
<westgard>lsitu: you can see all installed packages with "pip3 freeze"17:24
You will get a long list. In there you should see "scandir==1.5"17:25
<lsitu>westgard/dbernstein:: I see the yellow message: Ignoring scandir: markers 'python_version < "3.5"' don't match your environment. I’ll upgrade py to see how it goes. Thanks.
<westgard>lsitu: just curious, what is the output of 'python3 --version' for you?17:26
<lsitu>westgard: Python 3.6.017:27
<dbernstein>you should be fine.
<westgard>Yes, so in theory scandir in part of 3.6 and doesn't need to be installed.
<dbernstein>did you try running “sudo -H pip3 install -r requirements.txt .” (with period at the end?)17:29
* peichman leaves17:30
<dbernstein>lsitu: not sure if you saw my previous question.17:41
<lsitu>dbernstein: Sorry I just tried and I still got the same error.17:43
<dbernstein>lsitu: what os are you running?
<lsitu>dbernstein: OS X Version 10.9.517:44
* dbernstein leaves17:45
* dbernstein joins
lsitu: what happens if you try “python3 fcrepo_verify/cli.py /path/to/importexport.yml”17:48
<lsitu>dbernstein: It works with ‘python3 fcrepo_verify/cli.py ./importexport.yml’. Thanks.17:50
<dbernstein>lsitu: I just uninstalled and installed using pip3 and it still for me. I’m on osx 10.10.5 with python 3.6.117:55
what happens if you add “—upgrade” to the pip3 command?17:56
lsitu: you might also try “sudo -H pip3 uninstall -r requirements.txt . “ followed by “sudo -H pip3 install -r requirements.txt . “17:58
* manez leaves17:59
* whikloj leaves
* manez joins
<lsitu>dbernstein: It doesn’t work: Ignoring scandir: markers 'python_version < "3.5"' don't match your environment. And “—Upgrade” get error “no such option: -—”.18:02
* manez leaves18:04
* manez joins
* manez leaves18:05
<westgard>dbernstein, lsitu: It seems to me that if you've got python 3.6 then the verification tool should not be complaining about scandir.18:06
* manez joins
<dbernstein>westgard: I agree.18:07
<westgard>This would seem to be a problem in the verification tool, because if the python version is greater than 3.5, it should be importing scandir from the os module which is part of the standard library
So either the verification tool is somehow running against a different (lower) python, or there's an issue in our import logic18:08
<dbernstein>lsitu: there should be two dashes - looks like my irc client is fusing them into one.18:09
lsitu: which version of pip3 are you running? (pip3 —version)18:10
<lsitu>dbernstein: Yes, I added two dashes.
* mikeAtUVa leaves
<westgard>lsitu: sorry to pepper you with questions, but I wonder if the tool got installed on your system to run with python 2?18:11
Did you do "python3 setup.py install"
If you did "python setup.py install" (without the '3') then I think the installed tool might be running against your default python.18:13
<lsitu>dbernstein: Yes, I had ran “python setup.py install” at the very beginning.
<westgard>lsitu: try overwriting that by doing "python3 setup.py install" instead.18:15
lsitu: you can check what is installed with "cat /usr/local/bin/fcrepo-verify"18:17
<lsitu>dbernstein: Still the same warning after doing doing "python3 setup.py install”.18:18
<lsitu>westgard/dbernstein: $ cat /usr/local/bin/fcrepo-verify18:20
cat: /usr/local/bin/fcrepo-verify: No such file or directory
<westgard>Oh, probably because setup.py is failing due to the scandir problem
So it's never getting installed properly18:21
<lsitu>That makes senses.
westgard/dbernstein: Something interesting: I run “fcrepo-verify ./importexport.yml” and it wroks now, though “sudo -H pip3 install -r requirements.txt .” still warns “Ignoring scandir: markers 'python_version < "3.5"' don't match your environment”.18:25
And “cat /usr/local/bin/fcrepo-verify” still returns “cat: /usr/local/bin/fcrepo-verify: No such file or directory”.18:26
<westgard>Hmm. It seems like maybe it got installed elsewhere? Doing "which fcrepo-verify" will tell you where.18:27
<lsitu>Yep, it’s installed in “/Library/Frameworks/Python.framework/Versions/3.6/bin/fcrepo-verify”.18:28
<westgard>OK! That's some progress. So if you read that file you should see your python 3.6 version referenced near the top.18:29
That means fcrepo-verify will run with that version of python. And if it is 3.6, then you don't need scandir to be installed.18:30
The tool should import scandir from the os library (installed by default) on versions > 3.5
So the ignoring scadir response is actually correct for your version.18:31
<lsitu>It seems like “sudo -H pip3 install -r requirements.txt .” help though it throws the warning.18:32
<westgard>Possibly. Doing setup.py should install required modules, but dbernstein's command with sudo obviously has more power to force installs if there are permissions issues.18:34
Thanks for keeping with this, lsitu! I hope you'll be able to test now?18:36
<lsitu>westgard/dbernstein: Yes, I can run the veryfy tool now. Thank you both.18:38
<westgard>lsitu++ dbernstein++
* benpennell leaves18:56
* westgard leaves19:08
* rosiel joins19:14
* benpennell joins19:28
* westgard joins19:54
* benpennell leaves20:34
<awoods>dbernstein: commit hooks should now be enabled for fcrepo-import-export-verify20:38
* rosiel leaves21:17
* mikeAtUVa joins22:03
* westgard leaves22:15
* mcritchlow leaves22:20
* manez leaves22:28
* benpennell joins22:36
* westgard joins22:38
* lsitu leaves22:41
* github-ff joins22:43
[fcrepo-import-export] awoods closed pull request #87: Verify binary digests are used during import of Bags. (master...feature/verify_digest) https://git.io/vHIdu
* github-ff leaves
* github-ff joins22:47
[fcrepo-import-export] awoods deleted serialized-config at 339095e: https://git.io/vHO6z
* github-ff leaves
* mikeAtUVa leaves22:55
* github-ff joins22:57
[fcrepo-import-export-verify] jwestgard closed pull request #35: Adds bag support to the verification tool. (master...fcrepo-2408-bags) https://git.io/vHOik
* github-ff leaves
* westgard leaves23:09
* manez joins23:29
* awoods leaves23:30
* manez leaves23:34
* benpennell leaves00:28
* benpennell joins00:29
* manez joins00:30
* manez leaves00:34
* benpennell leaves00:45
* manez joins01:30
* manez leaves01:35
* manez joins02:32
* manez leaves02:36
* github-ff joins03:01
[fcrepo-import-export-verify] dbernstein opened pull request #36: Verification tool now timestamps logs and csv reports and places them in (master...fcrepo-2475) https://git.io/vHO5d
* github-ff leaves
<dbernstein>[Import/Export Standup]03:08
Finished yesterday:
* https://jira.duraspace.org/browse/FCREPO-2408
(make verification tool work with BagIt Bags)this morning
* https://jira.duraspace.org/browse/FCREPO-2475
(verifcation tool should log to timestamped log files)
Working on today:
(make verification tool handle versions)
* manez joins03:32
* manez leaves03:37