Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* thomz joins02:44
* ruebot leaves07:07
* ruebot joins07:09
* dwilcox joins07:49
* dwilcox leaves07:51
* dwilcox joins07:52
* dwilcox leaves07:56
* coblej joins08:02
* dwilcox joins
<ruebot>[Import/Export Standup]08:15
* Completed yesterday: - Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/8 - Tested and reviewed https://github.com/fcrepo4-labs/fcrepo-import-export/pull/9 - Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/10 - Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/11
- Tested and rviewed https://github.com/fcrepo4-labs/fcr4epo-import-export/pull/14
* Planning on completing today: - Test and merge https://github.com/fcrepo4-labs/fcrepo-import-export/pull/12 - Review and merge https://github.com/fcrepo4-labs/fcrepo-sample-dataset/pull/6 - Review Fedora 4 Import Export Utility Tests - Determine whether https://jira.duraspace.org/browse/FCREPO-2127 is good to close
* Blockers / Need help with: - None
[Import/Export Standup]
that came through ugly
* Completed yesterday:
- Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/808:16
- Tested and reviewed https://github.com/fcrepo4-labs/fcrepo-import-export/pull/9
- Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/10
- Tested and merged https://github.com/fcrepo4-labs/fcrepo-import-export/pull/11
- Tested and reviewed https://github.com/fcrepo4-labs/fcrepo-import-export/pull/14
* Planning on completing today:
- Test and merge https://github.com/fcrepo4-labs/fcrepo-import-export/pull/12
- Review and merge https://github.com/fcrepo4-labs/fcrepo-sample-dataset/pull/6
- Review Fedora 4 Import Export Utility Tests
- Determine whether https://jira.duraspace.org/browse/FCREPO-2127 is good to close
* Blockers / Need help with:
- None
* dhlamb joins08:25
* manez joins08:26
* westgard joins08:33
* acoburn joins08:39
* barmintor_ joins08:46
ruebot: out sick today and haven't slept, proably not on call08:47
<ruebot>barmintor_: :-(
barmintor_: hope you feel better man!
<barmintor_>ruebot: thx
* barmintor_ leaves08:48
* whikloj joins08:56
* justinsimpson joins08:58
<awoods>[Import/Export Standup]09:00
* Completed yesterday:
-- Code reviews
-- Add Standard Plugins: https://jira.duraspace.org/browse/FCREPO-2161
-- Fix broken executable jar: https://jira.duraspace.org/browse/FCREPO-2160
* Planning on completing today:
-- Create integration test framework: https://jira.duraspace.org/browse/FCREPO-215509:01
-- Use top-level dependency management: https://jira.duraspace.org/browse/FCREPO-2162
* Blockers / Need help with:
-- None
<escowles>[import/export standup]09:05
* yesterday: finished binary descriptions09:06
* today: export logging and error handling, maybe look at basic import functionality if barmintor's not working that today
* blockers: none09:07
* youn joins09:21
[Import/Export Standup]
* Completed yesterday: generated OWL data; uploaded to shared Google folder09:22
* Planning on completing today: script to load OWL files and binaries (thanks for pointing out the scripts on fcrepo-labs)
* Blockers / Need help with: none
* youn leaves
* bseeger joins09:26
* coblej leaves09:41
<awoods>westgard: ping
* coblej joins09:42
<justinsimpson>[Import/Export Standup]09:48
[Import/Export Standup]09:49
* Completed yesterday:
experimented with adding sample data to my fedora test instance
* peichman joins
<justinsimpson>tried running the export utility, but I didn't see any output
* Planning on completing today:
successfully run the export utility
looking at the sample data others have provided so far
contintue to make my own sample data
* Blockers / Need help with:
not sure yet how to use the samples youn put in the shared google folder
<bseeger>[Import/Export Standup]09:54
* Completed yesterday: PR code testing, fixing 214909:55
* Planning on completing today: more PR testing, not sure what else
* Blockers / Need help with: none
<westgard>[Import/Export Standup]10:04
* Completed yesterday:
- pull request for some sample data (200 objects)
* Planning on completing today:
- sort out some issues with sparql update errors in my sample data loader
- add external references to the sample dataset
* Blockers:
- Weltschmertz and the imponderableness of local master branches :-)
* thomz leaves10:08
* github-ff joins10:11
[fcrepo-import-export] escowles pushed 1 new commit to binary-descriptions: https://git.io/vimIx
fcrepo-import-export/binary-descriptions 2fe5499 Esmé Cowles: Supporting multiple descriptions for binaries
* github-ff leaves
<ruebot>westgard: you around?10:18
<coblej>Our production Fedora 3.8.0 repository is down ... could use some help if anyone is available who might be able to help
<ruebot>coblej: i got 10 minutes before i jump on a call, what are the errors?10:20
<coblej>starting Fedora web app hangs on or after (ResourceIndexModule) Configuring triplestore connector from fcfg datastore configuration localMulgaraTriplestore10:21
tried running server/bin/fedora-rebuild.sh but it displayed banner and then seemed to hang ... never got prompt about which index to rebuild10:22
would have expected that prompt fairly quickly ... but maybe not? it is a large-ish (360,000 object) repository10:23
earlier today, there was a problem with auto mounts on the server but server has since been rebooted and mounts appear to be okay now10:24
<ruebot>coblej: can you gist up some of the output? catalina.out and the output of what happens with fedora-rebuild.sh?10:26
<coblej>working on it10:28
^ is fedora-rebuild.sh output10:31
* youn joins
<coblej>with Tomcat (and thus presumably Fedora) stopped
^ server/logs/fedora.log output on restarting Tomcat
<westgard>ruebot: you are in the middle of somehting and also on a call (or so I gather), but in response you your earlier ping, yes I am around10:38
* github-ff joins
[fcrepo4] yinlinchen closed pull request #969: fcr:backup error response (master...FCREPO-1788-Update) https://git.io/vimYW
* github-ff leaves
<awoods>westgard: My earlier ping was about your plants.tar.gz... but we are resolving that in GitHub.10:39
* github-ff joins10:41
[fcrepo-import-export] escowles created logging (+1 new commit): https://git.io/vimOT
fcrepo-import-export/logging 441c3a3 Esmé Cowles: Using shared log config file
* github-ff leaves
* github-ff joins10:42
[fcrepo-import-export] escowles opened pull request #15: Using shared log config file (master...logging) https://git.io/vimO3
* github-ff leaves
* youn leaves10:47
<justinsimpson>awoods (or anyone else), can I ask what I assume is a dumb question about how to run the import export utility?10:57
when I run this command line:
* github-ff joins
[fcrepo-import-export] escowles pushed 3 new commits to master: https://git.io/vimso
fcrepo-import-export/master 1ac5ded bseeger: Removes trailing slash on resource, if present...
fcrepo-import-export/master 80925a0 bseeger: Cleanup based on feedback
fcrepo-import-export/master ff095f5 Esmé Cowles: Merge pull request #12 from bseeger/trailing-slash...
* github-ff leaves
<justinsimpson>java -jar fcrepo-import-export-driver/target/fcrepo-import-export-driver-0.0.1-SNAPSHOT.jar --mode export --resource http://localhost:8080/fcrepo/rest/booktest --binDir /tmp/test --descDir /tmp/test
I see output that says ' Exporting!' but nothing else. I assumed I would see content in /tmp/test10:58
<escowles>justinsimpson: the package name change broke the logging config — there's a PR to fix it: https://github.com/fcrepo4-labs/fcrepo-import-export/pull/1510:59
<justinsimpson>escowles, ok
<awoods>escowles: but it sounds like justinsimpson is getting no downloaded content.
<bseeger>justinsimpson: you don't see the exported desc/binary data at all in /tmp/test?
<justinsimpson>the /tmp/test directory is empty11:00
<ruebot>JIRA 404'ing for anybody else?
* awoods going into calls
<escowles>ruebot: i got that and then it worked when i tried again
<ruebot>escowles: weird. working for me again.
<bseeger>ruebot: it's do everything twice day11:01
ruebot: it's do everything twice day
escowles: i should be good to merge 14... but i'm getting on TravisCI fail, and one pass.
* youn joins11:03
* github-ff joins11:04
[fcrepo-import-export] escowles pushed 1 new commit to binary-descriptions: https://git.io/vimGy
fcrepo-import-export/binary-descriptions 93a7071 Esmé Cowles: Fixing Travis errors
* github-ff leaves
<escowles>ruebot: looks like two checkstyle errors (not sure why i'm not seeing them when i run the tests) — i just pushed a fix
afk # getting some lunch — talk to y'all in an hour
* github-ff joins11:09
[fcrepo-import-export] ruebot pushed 1 new commit to master: https://git.io/vimZz
fcrepo-import-export/master a21ed38 Esmé Cowles: Export binary descriptions (#14)...
* github-ff leaves
* manez leaves11:21
<ruebot>coblej: do y'all see anything relevant in localhost.2016-09-01.log ?11:22
coblej: or fedora.log?
* github-ff joins
[fcrepo-import-export] ruebot deleted binary-descriptions at 93a7071: https://git.io/vimc6
* github-ff leaves
* manez joins11:23
<justinsimpson>I have tried the logging branch (checked out logging branch, did mvn clean install, ran same command line) same results - no extra input to stdout, nothing in /tmp/test11:24
I installed Fedora using the vagrant box, so I have 4.5.111:25
* youn leaves
<coblej>ruebot: saw these errors earlier this morning in the localhost log file ... https://gist.github.com/coblej/993d0df9de9e45d814aa37d092fa37da
seem to indicate a problem with the resource index
ruebot: we are contemplating moving the current resource index aside for the moment and seeing if fedora-rebuild.sh will run and rebuild it11:27
does that seem like something worth trying?
* westgard leaves11:29
* youn joins11:32
* bseeger leaves11:34
<ruebot>coblej: backing up the current RI, moving it, and rebuilding? yeah. that's something i would give a shot if it were happening to me.11:36
coblej: other thing to check before you do that. you mentioned you had mount problems earlier. double-check all your directory permissions.
<coblej>ruebot: moved it but fedora-rebuild.sh still hangs11:37
after displaying the opening banner
even with a large repo, I presume I should see the prompt to rebuild resource or SQL index fairly quickly?11:38
<ruebot>coblej: give it bit. i've had to rebuild mine, and we're around 800k.11:42
<coblej>ruebot: how long "a bit" to produce the prompt? minutes, hours?
<ruebot>coblej: trying to remember... it's been awhile. maybe 10-15, see what happens? check htop too, see if you're getting any processes there.11:43
* acoburn leaves11:45
<justinsimpson>ruebot escowles, I got the exporter to work, I had to turn off http auth on my repo - is there a way to configure the exporter to use http auth?11:50
<escowles>justinsimpson: oh, we probably need a ticket for auth
<justinsimpson>I can make a ticket11:51
<justinsimpson>first reported fcrepo issue for me ever: https://jira.duraspace.org/browse/FCREPO-216311:53
not sure I did that right, feel free to correct
* bseeger joins
* mikeAtUVa joins12:11
<mikeAtUVa>is anyone talking?12:22
awoods is now
<ruebot>mikeAtUVa: awoods is
<mikeAtUVa>ruebot... yeah, had to reconnect. thanks.12:23
<bseeger>awoods: youn's dataset: https://drive.google.com/drive/folders/0B42p4oXygqHfS2J6MmRoNEQ4WEU12:24
<escowles>i opened one of the OWL files and it has URIs like "http://www.Department0.University0.edu/FullProfessor0"12:29
if those were something like "http://localhost:8080/rest/www.Department0.University0.edu/FullProfessor0" it could be loaded directly into fedora12:31
<bseeger>example from dataset: https://gist.github.com/bseeger/cf5171c40c1e512b1c3a8e19bfec24f5
snippet really
<mikeAtUVa>Is ruebot at an airshow?12:42
<escowles>mikeAtUVa: yes — yes he is
<bseeger>mikeAtUVa: he says he's at work...
<ruebot>mikeAtUVa: yeah, there is an air show in toronto every labour day weekend.12:44
mikeAtUVa: this twitter account sums it up perfectly https://twitter.com/Toronto_AirShow
* westgard joins12:46
<mikeAtUVa>It's not *just* that, it also guides the manual testing which is as/more important.12:52
<bseeger>awoods: 1:30 is fine w/ me12:54
<ruebot>you can squash in merge in the GitHub interface now13:00
communication is good :-)13:05
<escowles>westgard: here's a good blog post on squashing commits by jeremy friesen: http://ndlib.github.io/practices/one-commit-per-pull-request/13:07
<escowles>the shortcut is: if you want to rebase your branch for merging into master, you can do "git rebase -i master" to do that without having to count commits13:09
escowles: i did not know about that! i've been counting for years.
<escowles>ruebot: i've heard that from lots of people — i don't remember how i found out you can do the branch reference instead13:10
<justinsimpson>esmecowles++ nice tip
* tjohnson joins13:13
<escowles>could be nice to have general working hours and/or rough % of time in the sprint participants list13:14
pinging awoods bseeger justinsimpson mikeAtUVa ruebot westgard youn13:15
* md5wz__ joins13:16
* ruebot has irrsi behind screen on some random server... with hiligh plugin window.
* mikeAtUVa leaves13:17
<escowles>afk bbi 1 min13:18
* coblej leaves13:31
* dchandekstark joins13:33
<bseeger>I have to run - have a nice weekend13:40
* escowles leaves13:41
* escowles joins13:42
* bseeger leaves13:43
* coblej joins13:50
* escowles_ joins
<youn>I'll be flying to the West Coast tomorrow. I'll try to be here from 9 to 5 PT next week.13:54
* escowles leaves
<youn>Does anyone have a link to what is covered in integration testing that I could use to revise the test plan?13:58
<md5wz__>youn: integration testing is essentially when our software build routines spins up a running copy of the software we're testing and software with which it interacts (the import export tool and fedora4 in our case) and then performs scripted actions and verifies that the results are what would be expected.14:02
<coblej>ruebot: thanks for your help earlier today on our production problem ... I went to lunch and my colleagues figured it out in my absence ... a stale activemq lock file14:03
ruebot++ dchandekstark++ jackhill++
<md5wz__>youn: In our case, I'd expect we'd have a few representative resources (that exhibited all of the characteristics we'd want to verify were properly imported/exported) which we'd put into fedora then we'd export and test that our export files included all the values we expected, then we'd import them and test for the presence of values in fedora.14:04
youn: the reason it's not sufficient to verify the feature works is that we can't really predict what we've missed or failed to implement, so user testing (possibly with larger datasets) can flush out edge cases.14:05
youn: Not sure if that helps..
<ruebot>coblej: ohhhhhhhhhhhhhhh! man. i was going to mentioned yanking the activemq directory next. :-)
coblej: glad to hear y'all are good now.14:06
<coblej>glad to get my Labor Day weekend back14:07
* mikeAtUva leaves14:13
<youn>md5wz__: Thanks! Very helpful!14:19
Could the jmeter tests be used with the larger datasets if performance is a concern? (Performance is currently in the test plan.)14:24
<ruebot>youn: they could be used for benchmarking. but, i don't think they could be used for integration testing.14:26
* mikeAtUva joins14:28
<awoods>westgard: FYI, I have created three new tickets following on from your dataset creation:14:44
<westgard>awoods++ ruebot++
<ruebot>awoods: you good on https://github.com/fcrepo4-labs/fcrepo-import-export/pull/15 ? if so, i'll merge.
<westgard>Thanks for your help with the merge
<awoods>ruebot: #15 looks good... although I wonder if it would be even nicer if those particular logging statements went instead to a FileAppender instead of ConsoleAppender: http://logback.qos.ch/manual/appenders.html14:49
escowles_: thoughts ^^
<westgard>question regarding FCREPO-2167, adding links to external content14:50
<escowles_>it would probably be good to have at least some logging to the console to show progress, but maybe more verbose logging to a file?
<westgard>The objects in the sample dataset are related to a local project, in which context the images are served from our website via apache
So for example: http://www.lib.umd.edu/plantpatents/binaries/20200/pp20297.pdf14:51
<awoods>escowles: I was wondering if the logfile would an audit of the export: like what ruebot shows: https://github.com/fcrepo4-labs/fcrepo-import-export/pull/15#issuecomment-244402948
<escowles>that would be good14:52
<westgard>I was thinking one approach would be to create a version of the dataset in which half of the patents link to images stored as pcdm:File objects in the same Fedora, and half point to the external images on our website
<awoods>westgard: feel free to create a new dataset if that makes more sense.
westgard: that would work
<westgard>one caveat is that those external resources are not guarateed to be there forever14:53
<awoods>escowles: I don't want to hold you up... but if you think you can update your PR, great. Otherwise we can make a follow-on ticket.
<westgard>That is why they are going into Fedora
<awoods>westgard ;)
westgard: would it make more sense to link to more stable web URLs?
westgard: http://google.com ?14:54
<westgard>The alternative is the link to the USPTO database entry for each patent
But that's not an image resource, does that matter?
<awoods>westgard: anything stable is fine
<escowles>awoods: i'd prefer a follow-on ticket (or just keeping that JIRA issue open to handle import and export logging and expectations about what that should look like)
<awoods>westgard: but an image or a static document would certainly be best...14:55
westgard: especially if we are thinking about checksums.
<westgard>OK so it just needs to be a link to a resource on the web, not a link to an external binary resource
Ok I was typing before reading
<awoods>westgard: or ideally, a stable binary resource on the web
<westgard>Is the idea that the exporter is going to be confguraable to be able to grab those external resources?14:56
<ruebot>escowles, awoods: FCREPO-2153 would stay open after the merge, since we still need to do the import side.14:57
<westgard>In which case I do think the image is the better option.
<awoods>escowles: I am inclined to get your PR in, and have a follow-on... less loose ends.
<awoods>ruebot: I will merge #15 and do some JIRA gardening around it... ok?14:58
<westgard>I will go with my first idea, to link to images on our website. That will work for the immediate purpose of this project.
<awoods>westgard: sounds good14:59
<ruebot>awoods: sounds good!15:04
* github-ff joins
[fcrepo-import-export] awoods deleted logging at 441c3a3: https://git.io/vim74
* github-ff leaves
<ruebot>awoods: aww, you did the other merge type. escowles wanted a merge commit on that one.15:05
<awoods>ruebot: did the hash change?
ruebot: yes... the hash changed. Sorry, escowles, I was hoping that would not be the case. Lesson learned.15:06
<escowles>awoods: it's not a big deal — at most it's a minor annoyance if branches build up and i have to check what's been merged and what hasn't15:07
<justinsimpson>imort export sprint meeting minutes are on the wiki now - https://wiki.duraspace.org/display/FF/Import+-+Export+Sprint+Meetings page is starting to get pretty long, should today's meeting be it's own wiki page?15:11
<justinsimpson>ruebot not sure I got all the names in attendance right15:12
<ruebot>justinsimpson: just updated.15:13
justinsimpson: if need be, we can split them out into to their own pages at the end of the sprint.
<awoods>justinsimpson: using the {expand} macro can also help in the meantime.15:16
<justinsimpson>awoods ok, that might help
* awoods lunch15:17
* dchandekstark leaves15:25
<justinsimpson>is there a CLA for the Fedora project? I looked at the wiki and didn't see anything mentioned, so I assume there is not?15:34
nm there is I was looking in the wrong place15:35
<awoods>good man, justinsimpson15:40
<justinsimpson>its listed on the duraspace wiki, I didn't find a link in the fedora repository section
<awoods>justinsimpson: yes, this is the one: https://wiki.duraspace.org/display/DSP/Contributor+License+Agreements15:41
westgard ^^15:42
youn ^^
<justinsimpson>assuming contributions to sampledata repo should require cla , and anyways it is easier to sign it than to worry about it15:43
<awoods>justinsimpson: exactly. It would just be simplest to have it signed.
<youn>what about for LUBM; is it okay to cite in the readme?
<awoods>youn: I suspect so15:44
youn: what is their license?
* tjohnson leaves15:47
<youn>ruebot: I am finished with my edits to the test plan but am waiting to hear back from a prof on 1.b.ii. (It may be too strong. Unless someone here knows conclusively.)
awoods: I don't think it has one: http://swat.cse.lehigh.edu/projects/lubm/15:48
<youn>awoods: This is how it is cited by another project: https://github.com/rvesse/lubm-uba
* dwilcox leaves15:50
<awoods>youn: We can probably just reference it with a link to their site.
* dwilcox joins15:52
<ruebot>"link to their site"++16:01
<youn>The link is in a readme file packaged with the owl files. I have to leave now but will try to start on the script later this evening.16:05
* youn leaves16:06
* manez leaves16:09
* github-ff joins
[fcrepo-import-export] escowles created basic-import (+1 new commit): https://git.io/vimp2
fcrepo-import-export/basic-import efe6ae2 Esmé Cowles: [WIP] Partially-working import utility.
* github-ff leaves
<escowles>well, i didn't really get import working, but i did get something to compile and import some stuff, but there's still work to be done: https://github.com/fcrepo4-labs/fcrepo-import-export/compare/basic-import16:14
i figure it might be a good starting point next week, but it's no big deal if barmintor already got started and has something better already
in any event, i'm heading out for the day — have a great weekend everybody
<ruebot>escowles++ #have a good weekend!16:16
* dchandekstark joins16:26
* dchandekstark leaves16:31
* dhlamb leaves16:44
* escowles leaves
* escowles joins16:45
* peichman leaves16:52
* dwilcox leaves
* escowles leaves16:59
* dwilcox joins
* whikloj leaves17:01
* dwilcox leaves17:07
* coblej leaves17:26
* manez joins17:29
* manez leaves17:31
* manez joins17:32
* manez leaves17:47
* westgard leaves18:18
* dchandekstark joins18:28
* dchandekstark leaves18:32
* github-ff joins18:47
[fcrepo-import-export] awoods opened pull request #16: Add integration test framework (master...fcrepo-2155) https://git.io/viYGi
* github-ff leaves
* dchandekstark joins20:05
* dchandekstark leaves
* dchandekstark joins20:06
* dchandekstark leaves20:17
* jgpawletko joins21:37
* jgpawletko leaves21:40
* dchandekstark joins22:18
* dchandekstark leaves22:22
* mikeAtUva leaves22:51
* f4jenkins joins23:12

Generated by Sualtam