Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* dbernstein leaves04:06
* manez joins04:33
* manez leaves04:38
* manez joins05:34
* manez leaves05:38
* manez joins06:35
* manez leaves06:36
* manez_ joins
* manez_ leaves06:41
* mikeAtUVa joins07:07
* mikeAtUVa leaves07:11
* dhlamb joins07:13
* rosiel joins07:17
* manez joins07:37
* manez leaves07:41
* rosiel leaves07:50
* rosiel joins07:52
* rosiel leaves07:59
* manez joins08:00
* manez leaves08:02
* manez joins
* awoods joins08:04
* umgrosscol joins08:10
* youn joins08:19
* coblej joins08:29
awoods: I noticed that you closed https://jira.duraspace.org/browse/FCREPO-2455 (Import/Export: Document import structural expectations). The work that I did documenting the export format was intended as a partial response to that but I don't think it addressed everything the requester (Bridget) was looking for. She was also requesting documentation about the required "LDP containment structure". That is probably beyond my ken (n08:35
ot even sure I fully understand what she's asking for there). Anyway, I just wanted to note that I'm not sure what I did addressed all of the concerns in the issue.
* manez leaves08:37
* umgrosscol leaves08:39
* umgrosscol joins08:40
<youn>[Import/Export Standup]08:42
<coblej>[Import/Export Standup]08:43
Finished yesterday:
Document export format in partial response to https://jira.duraspace.org/browse/FCREPO-2455 (Import/Export: Document import structural expectations)
Working on today:
Testing sprint changes using interim code releases
<youn>Finished yesterday:
- tested export with lubm
- regenerated lubm backup, replacing link for external content; https://yale.box.com/s/0rkv4q1akg7g1w3oq4zhnrkafeppmplg needs to be added to github repo; previous versions should be deleted
Working on today:
- test lubm import and verify export and import
- test versions and bags
- https://jira.duraspace.org/browse/FCREPO-2473 (verification tool should handle import-export v1.1 config files)08:44
- none
* dwilcox joins08:46
* dwilcox leaves08:48
* dwilcox joins
* dwilcox leaves08:50
* github-ff joins08:52
[fcrepo-import-export] escowles pushed 1 new commit to find-root-fallback: https://git.io/vH38E
fcrepo-import-export/find-root-fallback 0fab8b4 Esmé Cowles: Moving tests to unit test
* github-ff leaves
* benpennell joins08:53
* umgrosscol leaves
* umgrosscol joins08:54
* benpennell leaves08:55
<escowles>[Import/Export Standup]
* Trailing slash issues https://jira.duraspace.org/browse/FCREPO-2467
* Containment predicates don't work when pointed at /rest https://jira.duraspace.org/browse/FCREPO-2474
* Test import into Cavendish https://jira.duraspace.org/browse/FCREPO-2476
* findRepositoryRoot should have a sensible fallback https://jira.duraspace.org/browse/FCREPO-2477
* Error importing export with inbound references https://jira.duraspace.org/browse/FCREPO-2483
* Squashing bugs from testing
* Couple meetings
* benpennell joins
<escowles>* None
<youn>Can authorization be set in the config file for export? If so, what is the name of the key? Thanks!08:59
What is the easiest way to configure authorizations with the one-click jar? Thanks!09:02
* manez joins09:03
* coblej leaves
* dwilcox joins09:06
* github-ff joins09:09
[fcrepo-import-export-verify] jwestgard pushed 1 new commit to master: https://git.io/vH3BJ
fcrepo-import-export-verify/master 839cb8b dbernstein: Verification tool now timestamps logs and csv reports and places them in (#36)...
* github-ff leaves
* westgard joins09:10
* rosiel joins
* mikeAtUVa joins
<westgard>awoods: quick process question -- after review and squash/merge, I put a reference to the commit that resolves the PR on the JIRA ticket. Do I also hit "close issue" or do you prefer to look it over and do that bit?09:11
<youn>westgard: I am going to unassign myself from https://jira.duraspace.org/browse/FCREPO-2473 (verification tool should handle import-export v1.1 config files). Is this okay? I don't think I'll have time to get to it.09:12
<westgard>youn: OK no problem. I think we are getting close. @dbernstein is a master of ticket whack-a-mole.
* github-ff joins
[fcrepo-import-export] escowles pushed 1 new commit to find-root-fallback: https://git.io/vH3BB
fcrepo-import-export/find-root-fallback 2faf170 Esmé Cowles: Removing Guava, improving JavaDocs
* github-ff leaves
<awoods>westgard: feel free to close the issue at that point.09:13
* manez leaves
<westgard>awoods: thanks will do
<youn>westgard, dbernstein++
<westgard>I will look at FCREPO-247 now.09:14
* whikloj joins09:15
<awoods>coblej: I agree that Bridget's comment around LDP containment structure is a little mysterious. The structure that you outlined is effectively the "LDP structure". There is a lot more detail that could go into describing the triples within each resource, but I think what you have created is all we should do for now.
Import/Export Standup]09:18
Finished yesterday:
- Reviewed tickets
Working on today:
- Reviewing tickets
- None
<mikeAtUVa>[import/export sprint standup]09:19
Finished Yesterday:
- https://jira.duraspace.org/browse/FCREPO-2464 documented "relaxed" triple mode
Working on Today:
- testing scenarios
- comments for https://github.com/fcrepo4-labs/fcrepo-import-export/pull/88
I will be out tomorrow afternoon and will miss the sprint retrospective. I can e-mail my thoughts and feedback if you'd like, awoods.
<awoods>youn: can you create a ticket for "regenerated lubm backup, replacing link for external content;..."
mikeAtUVa: that would be helpful
mikeAtUVa: It would also be good to get your thoughts on where things stand today... so that we have time to tie any loose ends.09:20
* rosiel leaves09:21
* coblej joins09:23
<youn>awoods: https://jira.duraspace.org/browse/FCREPO-2488 (thanks!)09:26
* manez joins09:28
<westgard>[Import/Export Standup]09:29
Finished Yesterday:
https://jira.duraspace.org/browse/FCREPO-2449 (handle all responses from Fedora)
Reviewed several tickets
Working on today:
https://jira.duraspace.org/browse/FCREPO-2473 (use v1.1. config files, including working with inbound references)
bug squashing (https://jira.duraspace.org/browse/FCREPO-2485, https://jira.duraspace.org/browse/FCREPO-2486) and/or ticket review
<mikeAtUVa>awoods: as for where things stand today if the export of versions PR is merged and the import of versions ticket is finished and merged by the end of the sprint, I think we'll be in great shape from my point of view. In a short while when 4.7.4 drops I'll gladly do the work of updating the tool and verify that it indeed meets our local requirements (switching away from fcr:backup and restore if feasible)
<awoods>westgard: when you close issues, try to ensure that the "Fix Version" is set to "fcrepo-importexport-0.2.0"
<westgard>awoods: OK sorry
<awoods>thanks, mikeAtUVa
westgard: sorry?? I have not noticed that not being the case... but was just mentioning it as a consideration.09:30
* yamil joins
* manez leaves
<mikeAtUVa>awoods: as for https://jira.duraspace.org/browse/FCREPO-2479 (indirect container relationships not showing up when viewing versions), I've no love of indirect containers, but I suspect it doesn't actually prevent the kind of backup and restore I intend to do with import/export, and I don't think the actual relationships are lost (ie, if you reverted to that version, even after a restore, I think the relationships would be presented)09:31
<westgard>awoods: consider it apology pre-payment for when I forget later
<awoods>westgard ;)
* rosiel joins09:32
<awoods>mikeAtUVa: maybe some testing and a little documentation would be helpful to clarify what is happening in the case of versioning indirect containers.
<benpennell>mikeAtUVa: you are correct, if you revert to a version the relationships will show up. It just presents problems for anyone interacting with it while its still a version
* manez joins09:33
<westgard>question for import/exporters: does export without binaries filter the RDF to remove links to the binaries, or does it just omit non-RDFsource resources from the export?
<mikeAtUVa>benpennell: to be clear are those relationships to resources that are or arent' part of the versioned tree?
<benpennell>mikeAtUVa: in my testing they were outside of the versioned tree09:34
<westgard>I'm asking this in reference to https://jira.duraspace.org/browse/FCREPO-2486
<benpennell>that's where my awkward initial "external resources" language on the ticket came from, couldn't think of a good way to phrase it like you just did :)09:35
<mikeAtUVa>benpennell: then there may be real problems, ie, if you restore a version and those resources don't exist the links won't exist anymore...
<awoods>westgard: the triples that reference binaries are removed09:36
<westgard>awoods: is that a new feature?
<mikeAtUVa>benpennell: because we have referential integrity between resources, the triples representing those links aren't *really* a part of the resource and aren't versioned.
<awoods>westgard: new this sprint
<benpennell>mikeAtUVa: i haven't tried taking away the referenced resources when restoring, so maybe so
<westgard>Ahh. OK. I thought this used to work and now it doesn't so good to know.09:37
<mikeAtUVa>benpennell: I'm pretty much in the "never delete anything" camp.
benpennell: LDP doesn't seem to work if resources disappear. I wonder if the links persist to a tombstone.09:38
* dhlamb leaves
<benpennell>mikeAtUVa: that'd be interesting to find out, does fedora normally switch references over to tombstones after a delete?09:40
[Import/Export Standup]09:44
Finished yesterday:
* Created ticket for issue with generated relations in versions https://jira.duraspace.org/browse/FCREPO-2479
Working on today:
* Respond to feedback on version export PR
* Continue to work on initial implementation of version import ticket
* Can't guarantee version import will be done by the end of the sprint, it will depend on how smoothly today goes and feedback on the export, but I hope to have a first pass at it
* none
* peichman joins09:45
* lsitu joins09:46
[Import/Export Standup]09:52
Finished yesterday:
Testing export and import the plant patent dataset (restored from plants.tar.gz) with options -b, -g (default/aptrust), -i, -p, -t, -x in https://wiki.duraspace.org/display/FF/Import+-+Export+Testing+Recipes
Working on today:
Use the verification tool to verify import/export.
Fix bugs that coming up from from testing.
It seems that the verification tool works fine for binary export at this time but has limitation on other options. What can we test now?
* dhlamb joins09:58
<awoods>lsitu: would you please update the table in the wiki with your test experiences? https://wiki.duraspace.org/display/FF/Import+-+Export+Testing+Recipes10:00
<lsitu>awoods: Sure.
<awoods>westgard: It looks like lsitu has a question for you in his "Blockers" section
lsitu: thanks
* mikeAtUVa leaves10:04
<westgard>lsitu: yes, the issue you identified with filtering RDF when exporting with no binaries means that RDFsources will show a mismatch in those cases.
lsitu: also, inbound refs has not yet been implemented.10:05
<awoods>westgard: would it make sense to flag types of "verification errors"? so that the user can see that the mismatches are "binary reference mismatches", for example.10:06
<westgard>lsitu: import should work against a Fedora that has the new code for updating server managed triples. Could you test that, perhaps?
<awoods>lsitu: https://s3.amazonaws.com/f4-artifacts/fcrepo-webapp-4.8.0-SNAPSHOT-jetty-console-a07a1a46.jar10:07
<lsitu>westgard: I’ll try it later. Thanks.10:08
<westgard>awoods: yes, that would make sense. Right now we're going to be in catch-up mode. There's quite a lot that has changed and I'm not sure we'll get everything fixed in the next 48 hrs.10:09
<awoods>westgard: I believe in you... and the team
<westgard>awoods: you are the wind beneath our wings...10:10
* peichman leaves
* awoods getting misty10:11
<lsitu>westgard: Will the verification tool work with different context like /f4 and /fcrepo for import? I got “AttributeError: 'FedoraResource' object has no attribute 'ldp_type’” error when importing source to http://localhost:8080//f4/rest that are exported from http://localhost:8080/fcrepo/rest.10:14
<westgard>lsitu: I think it should. You may have surfaced a different bug (one that I suspect I created yesterday).10:16
lsitu: could you file the error your observing as a JIRA bug ticket?10:17
* peichman joins
<lsitu>westgard: No problem.
<benpennell>mikeAtUVa awoods: More version woes. Export of old versions won't have generated membership relations, and therefore won't export the referenced objects. Then when importing there is a good chance that you would have references to objects that don't exist.10:18
mikeAtUVa awoods: Even if that were fixed, when replaying snapshots a version might reference something that no longer exists or exists in a later snapshot of another object being imported. Ordering all snapshots in the export chronologically might help some, but wouldn't help with the deleted resource issue. That does seem like the only way to figure out when to add proxies in though.
indirect containers are apparently not my friend after all10:19
awoods mikeAtUVa: so i'm not sure we can guarantee referential integrity when importing versions if indirect containers are involved10:22
<awoods>benpennell: sounds like a topic that could use some discussion... maybe on today's tech call?10:23
<benpennell>awoods: yes that would be helpful10:24
* rosiel leaves10:26
* manez leaves10:32
* manez joins10:46
* github-ff joins10:56
[fcrepo-import-export-verify] jwestgard opened pull request #37: update README to reflect current state of tool (master...readme) https://git.io/vH3KD
* github-ff leaves
* ruebot is here
* coblej here11:01
* coblej prepared to take notes
* escowles is here11:02
* escowles is the only one here? am i on the wrong call...?11:03
* escowles is actually here now11:04
* manez leaves
<escowles>though it sounds like a vibrating phone to me...
<dhlamb>really? pretty sure that's a vibrating phone. you guys have active imaginations
* zimeon joins11:18
* rosiel joins11:25
* youn leaves
* apb18 joins
* manez joins11:27
<whikloj>awoods: +111:29
awoods: I'll get some testing done, possibly with MySQL11:30
* mcritchlow joins
* mikeAtUVa joins11:31
* manez leaves11:33
* manez joins11:35
<whikloj>dhlamb: got your back11:43
awoods: I can take a sprint to my boss, probably get time for that11:44
* manez leaves
* manez joins11:47
<ruebot>apb18 lead that effort, right?
adopters guide?
* rosiel leaves
<apb18>"adopters guide"11:48
* rosiel joins11:50
<apb18>At some point, I'll have time to continue the adopters guide11:52
* rosiel leaves11:59
<zimeon>consecutive calls.... gotta dash, ciao12:00
<escowles>same for me
<whikloj>cheers all12:02
* github-ff joins12:07
[fcrepo-import-export] awoods closed pull request #88: Added support for lossless roundtripping of RDF and a mode to disable… (master...FCREPO-2461) https://git.io/vHtFk
* github-ff leaves
* coblej leaves12:09
* zimeon leaves12:17
<benpennell>mikeAtUVa escowles awoods: Case 1 (Version indirectly references a deleted resource):12:19
Resource1 has an indirect pcdm:hasMember relation to resource2. Create version1 of Resource1. Resource2 and related proxies are deleted, pcdm:hasMember goes away. Export Resource1. Import Resource1, stepping through versions. When it tries to import the first version, it references resource2 which no longer exists and would generate an error
mikeAtUVa escowles awoods: Case 2 (version indirectly references a resource that only exists in a specific version of another resource):12:21
Resource2 contains Child1. Create version1 of Resource2. Resource1 has indirect pcdm:hasMember relation to Child1. Create version1 of Resource1. Delete Child1. Export Resource1 and Resource2. Now when importing them, depending on the order in which versions are imported, Child1 may not exist when the version of Resource1 that references it is imported.
case 2 is complicated by us not being able to provide timestamps for versions on import, so you might not be able to import versions across objects in the right order after the first import.12:22
does this make any more sense?
* manez leaves12:23
* manez_ joins
<benpennell>mikeAtUVa escowles awoods: if indirect containers are taken out of the equation, import of versions is vastly simpler. i'm a little bit worried about the general recoverability of versions when this container type is used12:26
* github-ff joins
[fcrepo-import-export] escowles created filter-inbound (+1 new commit): https://git.io/vH3NZ
fcrepo-import-export/filter-inbound c446782 Esmé Cowles: Filtering out inbound links from serialized RDF
* github-ff leaves
* manez_ leaves12:29
* rosiel joins12:31
* dbernstein joins12:32
* manez joins12:34
<mikeAtUVa>benpennell: I'd say it works for me, but I should go see if those hydra folks using my repo are foolishly using indirect containers.12:38
<benpennell>mikeAtUVa: spoilers, they probably are12:39
<awoods>mikeAtUVa: they definitely are
benpennell: I will be reviewing your PR#85 @2:30pm ET... but as a heads-up, it has merge conflicts.12:40
<benpennell>mikeAtUVa: i'm thinking of removing them from unc's implementation now though
<escowles>benpennell: if you export and import a version of an indirect container, and then restore the version, do the membership triples then get generated correctly?12:41
* mcritchlow leaves
<benpennell>escowles: there's no importing of versions yet
* rosiel leaves
<benpennell>escowles: if you restore a version of an object that has an indirect containers, the membership triples do appear12:42
<escowles>but assuming we implement it as it is now, do you think that would work? is this just a fedora bug of not generating those triples for versions of indirect containers?
<benpennell>escowles: yes i'm pretty sure those memberships relations would appear after importing, assuming what they referenced still existed12:43
the bug i was describing on the call is related, but not the direct cause of the issues in the cases i outlined above12:44
escowles: although there is also the issue of how the import tool loads all membership resources at the end, so they would not make it into imported versions without some refactoring12:46
* rosiel joins12:47
* umgrosscol leaves12:48
<benpennell>escowles: my understanding of how the version import would work (without indirect containers) is that it would locate the oldest version of a resource at import time, and import it as if it were the live object. It would then create a version of that in fcrepo, and load the next oldest version, create a version, etc, until it ran out of versions, at which point it would import the non-version copy of the resource
<mikeAtUVa>benpennell: that sounds good. If there are bugs or edge cases that seem like the won't work with that approach, they likely don't really work in fedora anyway.12:50
* rosiel leaves12:52
<benpennell>mikeAtUVa: while i'd like to work it that way, i'm not sure if its valuable if it doesn't work for a common use case like indirect membership relations?
mikeAtUVa: it seems like it would mean that no hydra instances could reliably import versions12:54
<mikeAtUVa>benpenell: that would obviously be a failure to implement lossless roundtripping for the masses, but is it your impression that the fix would require a dramatically different approach to importing or a change to the way fcrepo4 works?12:56
benpennell: unless you believe all the work would be in the wrong direction, there's certainly value in getting "half-way" there by having working import code that doesn't work in that (unfortunately common) scenario.12:58
<escowles>i agree: that approach seems reasonable to me, and would then let us figure out if there are fedora bugs that need to be fixed, or more things the importer needs to do to get the rest working12:59
* manez_ joins
* manez leaves13:00
* github-ff joins13:01
[fcrepo-import-export] escowles opened pull request #93: Filtering out inbound links from serialized RDF (master...filter-inbound) https://git.io/vH3hS
* github-ff leaves
* lsitu leaves
* lsitu joins13:02
* mcritchlow joins
<benpennell>mikeAtUVa escowles: i would mainly worry that people might try to use the feature without being aware of that limitation. but if that isn't a major setback, i can look at continuing with an implementation that doesn't try to address the complexities of integrity with indirect containers
I don't currently think that wouldn't in itself require a change to the way fedora works, although there may still need to be some considerable reworking of the behavior of the import tool13:03
well, the implementation of it, not really the outcomes of it
<escowles>benpennell: i think it's ok, at this stage, to work on features knowing they will be flawed — we're still in a state of development where some significant features aren't working and (i hope) it's clear that this tool isn't ready for production use yet
<benpennell>mikeAtUVa escowles: okay i'll keep looking into it this afternoon, PR 85 permitting13:05
<mikeAtUVa>benpennel: we're in fcrepo-labs... it's different than fcrepo4.
* umgrosscol joins13:08
* coblej joins13:10
* coblej leaves13:12
* coblej joins
* manez_ leaves13:16
* jgpawletko joins13:19
* jgpawletko leaves13:22
* manez joins13:23
* github-ff joins13:26
[fcrepo-import-export] lsitu opened pull request #94: Improved logging for HTTP validation and updated comment on params to avoid warnings. (master...feature/logging) https://git.io/vHsUX
* github-ff leaves
<benpennell>awoods: rebased my PR from master again, so it should be mergeable once more13:27
* manez leaves13:28
* manez joins13:31
* coblej leaves13:36
<westgard>can someone explain what is LegacyMode in the import/export config?13:40
* awoods on a call... but mikeAtUVa knows13:42
<westgard>mikeAtUVa: Could you clarify the meaning of the legacyMode parameter in import/export?13:43
<whikloj>westgard: My understanding is that in legacyMode you are _not_ allowed to overwrite the created and last modified properties.13:44
<westgard>whikloj: Thanks. I guess that means for now that if legacyMode is true, the verification tool will not be able to verify rdf on import because of server-managed triples.13:45
Or more precisely, it will throw an error for each RDFsource.13:46
<whikloj>westgard: I believe that mikeAtUVa skips the created and lastmodified if legacyMode == true because of the reasons you mention13:47
westgard: oh sorry you are talking about the verification tooling
westgard: I'm out of the loop but is the verification tooling still operating off the import/export config?13:52
<westgard>whikloj: yes, we use the config13:54
whikloj: I'm in the process of updating the verifcation tool to take into account all the current options
* manez leaves13:55
<westgard>whikloj: whether something was exported in legacyMode doesn't really matter for verification except to know what to expect in terms of behavior of server managed triples13:56
<mikeAtUVa>westgard: sorry, didn't see your comment until now... yeah, legacyMode affects the importer by stripping some triples on import.
<westgard>mikeAtUVa: Thanks. In an ideal world the verification tool would also act on that option by stripping those same triples, to avoid the mismatches that we currently get, but I think that's a bridge too far for this sprint.13:58
* youn joins
* umgrosscol leaves14:04
* youn leaves
* youn_ joins14:05
I would think that only the intrinsic properties of a resource, such as time created, would be specific to a version and that extrinsic relational properties would pertain to the resource across versions.14:09
<mikeAtUVa>youn_: that makes sense to me, and in that light, versioning is a more tractable problem.14:12
While versions currently contain all the resources contained by the versioned resources, other relationships (other than ldp:contains) aren't followed.
We'd be in better shape if versioning *didn't* cascade through ldp:contains, and in fact that might be where we have to go to make it sensible and aligned with the spec in the future.14:13
<youn_>I agree that ldp:contains should not cascade. This may be naive but I think versions should be substitutable. For example, would you want versions to have different authorizations?14:16
* coblej joins14:18
<youn_>What I meant is that it seems wrong for versions to have different LDP relationships.14:20
<mikeAtUVa>youn_: in a world where version snapshots were restricted to a single version, I think I agree with you (though I'm not versed on all the LDP relationships)14:23
er... snapshots were restricted to a single RESOURCE (not version)
<lsitu>awood/westgard: I can’t test verify import with updated server managed triples. I am still getting the AttributeError from the latest codebase on master even using the same context. See https://jira.duraspace.org/browse/FCREPO-248914:37
* youn joins14:42
* youn_ leaves14:44
* youn leaves14:51
* youn joins14:58
* coblej leaves15:00
<dbernstein>lsitu: a note: you may need to uninstall the tool and then reinstall to make sure you have the latest changes:
ie: “sudo -H pip3 uninstall fcrepo-verify; sudo -H pip3 install -r requirements.txt .;”15:01
* coblej_ joins15:02
<westgard>lsitu dbernstein: I think there's a bug in the FedoraResource class.
All these things are on my radar, but I can only work one thing at a time.
I think we are in the same boat as last sprint, where a number of issues get surfaced at the end because we are developing against a moving target15:03
<dbernstein>oh - okay. Still - it is important to uninstall before reinstalling when testing a new version. (I don’t believe this is an issue when invoking the tool via “python3 fcrepo_verify/cli.py /path/to/config.yml”)15:04
<lsitu>dbernstein: Yes, I ran “sudo -H pip3 uninstall -r requirements.txt .”, then “sudo -H pip3 install -r requirements.txt .” and then “python3 setup.py install”.15:06
* youn leaves15:07
* coblej_ leaves15:09
<dbernstein>lsitu: note: sudo -H pip3 uninstall -r requirements.txt . will not uninstall fcrepo-verify. You need to specify fcrepo-verify on uninstall: ie “sudo -H pip3 uninstall fcrepo-verify”15:10
<lsitu>dbernstein: But I see the latest code for bags works though the verify failed for 4.7.2 data:15:11
2017-05-25 11:56:01,754 INFO Running verification on Fedora 4 export
2017-05-25 11:56:01,754 INFO Verifying bag...
2017-05-25 11:56:04,524 INFO bag is valid :)
2017-05-25 11:56:04,524 INFO Commencing resource verification...
2017-05-25 11:56:14,524 INFO Verified 257 resources: successes = 36, failures = 221
2017-05-25 11:56:24,525 INFO Verified 512 resources: successes = 36, failures = 476
2017-05-25 11:56:34,524 INFO Verified 807 resources: successes = 121, failures = 686
* umgrosscol joins
<lsitu>2017-05-25 11:56:40,692 INFO Verified 1005 resources: successes = 200, failures = 805
2017-05-25 11:56:40,692 INFO Verification complete
<dbernstein>lsitu: yes bag validation just verifies that the bag is internally consistent and valid. The verification tests a source against a destination.15:14
lsitu: were you able to get this far before? Or are the errors you were seeing coming up in th logs?15:15
* ruebot leaves15:17
<lsitu>dbernstein: No. The bads export verify doesn’t work previously so I believe the tool is updated to the the latest version on master. Something must be wrong with import verify and it even failed with the same context for resource and source.15:18
* ruebot joins
* ruebot leaves
* ruebot joins
* ruebot leaves
* ruebot joins
<dbernstein>awoods or anyone who knows: which version of fcrepo4 should we be using for import/export tests?15:22
<awoods>dbernstein: https://wiki.duraspace.org/display/FF/2017-05+Import+-+Export+Sprint+04+Meetings#id-2017-05Import-ExportSprint04Meetings-Resources
<lsitu>dbernstein: Here is the log that could give you some hints:15:23
<awoods>dbernstein: basically master
* manez joins
<lsitu>2017-05-25 11:30:06,113 INFO Loading configuration options from ./importexport.yml
2017-05-25 11:30:06,114 INFO key (mode) and (import)
2017-05-25 11:30:06,114 INFO key (predicates) and (http://www.w3.org/ns/ldp#contains)
2017-05-25 11:30:06,114 INFO key (resource) and (http://localhost:8686/f4/rest)
2017-05-25 11:30:06,114 INFO key (source) and (http://localhost:8080/fcrepo/rest)
2017-05-25 11:30:06,114 INFO key (dir) and (/Users/lsitu/Documents/Fedora_Import_Export/data_1b)
2017-05-25 11:30:06,114 INFO key (binaries) and (True)
2017-05-25 11:30:06,114 INFO key (rdfLang) and (text/turtle)
2017-05-25 11:30:06,114 INFO Testing connection to http://localhost:8686...
2017-05-25 11:30:06,126 INFO Connection successful.
2017-05-25 11:30:06,127 INFO Starting verification...
2017-05-25 11:30:06,127 INFO Running verification on Fedora 4 import
2017-05-25 11:30:06,127 INFO Commencing resource verification...
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/bin/fcrepo-verify", line 11, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fcrepo_verify-0.2.0_SNAPSHOT-py3.6.egg/fcrepo_verify/resources.py", line 76, in is_binary
return self.ldp_type == LDP_NON_RDF_SOURCE
AttributeError: 'FedoraResource' object has no attribute 'ldp_type'
* umgrosscol leaves15:24
<lsitu>dbernstein: Note that it’s the latest fcrepo_verify-0.2.0_SNAPSHOT.15:26
<awoods>benpennell: thanks for the version export work. It looks great.15:28
benpennell: There are just a few minor questions/suggestions15:29
escowles: https://github.com/fcrepo4-labs/fcrepo-import-export/pull/92/files#r11856652815:35
* coblej joins
* coblej_ joins15:37
* coblej leaves
* umgrosscol joins15:38
* github-ff joins
[fcrepo-import-export] awoods pushed 1 new commit to master: https://git.io/vHsoE
fcrepo-import-export/master bed4e57 lsitu: Improved logging for HTTP status validation and updated comment on (#94)...
* github-ff leaves
* coblej_ leaves15:39
<escowles>awoods: i'll push a new commit to remove guava
* coblej joins15:40
* github-ff joins
[fcrepo-import-export] escowles pushed 1 new commit to find-root-fallback: https://git.io/vHso7
fcrepo-import-export/find-root-fallback 89823cb Esmé Cowles: Removing Guava
* github-ff leaves
* coblej leaves15:43
* coblej joins
* github-ff joins15:45
[fcrepo-import-export] escowles created if-unmodified-since (+1 new commit): https://git.io/vHsKB
fcrepo-import-export/if-unmodified-since 0afe480 Esmé Cowles: Adding If-Unmodified-Since header to PUT-to-update requests
* github-ff leaves
<awoods>escowles: looks like merge conflicts: https://github.com/fcrepo4-labs/fcrepo-import-export/pull/92
* github-ff joins15:46
[fcrepo-import-export] escowles opened pull request #95: Adding If-Unmodified-Since header to PUT-to-update requests (master...if-unmodified-since) https://git.io/vHsK2
* github-ff leaves
* westgard leaves
* github-ff joins15:50
<escowles>awoods: i've just rebased and updated https://github.com/fcrepo4-labs/fcrepo-import-export/pull/92
<github-ff>[fcrepo-import-export] escowles force-pushed find-root-fallback from 89823cb to 5580837: https://git.io/vHs6e
fcrepo-import-export/find-root-fallback 3ec5761 Esmé Cowles: Making findRepositoryRoot more robust
fcrepo-import-export/find-root-fallback d6500d9 Esmé Cowles: Moving tests to unit test
fcrepo-import-export/find-root-fallback 916d0c3 Esmé Cowles: Removing Guava, improving JavaDocs
* github-ff leaves
<awoods>thanks, escowles
* github-ff joins15:53
[fcrepo-import-export] escowles force-pushed if-unmodified-since from 0afe480 to 3c60ea5: https://git.io/vHs6W
fcrepo-import-export/if-unmodified-since 3c60ea5 Esmé Cowles: Adding If-Unmodified-Since header to PUT-to-update requests
* github-ff leaves
* github-ff joins
[fcrepo-import-export] awoods pushed 1 new commit to master: https://git.io/vHs6R
fcrepo-import-export/master 7081ed5 Esmé Cowles: Making findRepositoryRoot more robust (#92)...
* github-ff leaves
* github-ff joins16:50
[fcrepo-import-export] escowles created write-config (+1 new commit): https://git.io/vHsSK
fcrepo-import-export/write-config 6913a86 Esmé Cowles: Only write config files if the --writeConfig option is specified
* github-ff leaves
* github-ff joins16:51
[fcrepo-import-export] escowles opened pull request #96: Only write config files if the --writeConfig option is specified (master...write-config) https://git.io/vHsSy
* github-ff leaves
* rosiel joins17:06
* dwilcox leaves17:10
* coblej leaves17:16
* mikeAtUVa leaves17:18
* github-ff joins
[fcrepo-import-export] awoods closed pull request #85: FCREPO-2458 Export versions (master...export-versions) https://git.io/vHkPr
* github-ff leaves
* umgrosscol leaves17:19
* peichman leaves17:20
* whikloj leaves17:26
<dbernstein>awoods: I’m hunting around for the link to the testing page for the import-export sprint. Can you or anyone drop it in here.17:27
<awoods>dbernstein: https://wiki.duraspace.org/display/FF/Import+-+Export+Testing+Recipes17:28
<dbernstein>gracias awoods.
* lsitu leaves17:30
* escowles leaves17:47
* escowles joins17:48
* yamil leaves17:50
* escowles leaves17:52
* escowles joins17:56
* lsitu joins18:07
* dhlamb leaves18:11
* lsitu leaves18:54
* apb18 leaves19:01
* mcritchlow leaves
* lsitu joins19:03
* dwilcox joins19:35
<lsitu>awoods/escowles: With the latest codebase from maste, this one seems like a blocker: https://jira.duraspace.org/browse/FCREPO-2485.19:43
* github-ff joins20:25
[fcrepo-import-export-verify] jwestgard opened pull request #38: FCREPO-2473 (master...fcrepo-2473) https://git.io/vHGqC
* github-ff leaves
* benpennell leaves20:33
* dhlamb joins20:43
* dwilcox leaves20:45
* youn joins20:55
* youn leaves21:02
* rosiel leaves21:10
* dhlamb leaves22:52
* manez leaves22:56
* benpennell joins23:03
* awoods leaves23:11
* benpennell leaves23:16
* lsitu leaves23:19
* manez joins23:57
* manez leaves00:01
* manez joins00:58
* manez leaves01:02
* manez joins01:59
* manez leaves02:03
* dbernstein leaves02:04
* manez joins03:00
* manez leaves03:04