Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* jrgriffiniii joins00:14
* thomz joins02:55
* jrgriffiniii leaves04:04
* jrgriffiniii joins04:15
* apb18 joins08:16
* jrgriffiniii leaves08:29
* peichman joins08:30
* dwilcox joins08:32
* peichman leaves08:33
* arebenji joins08:49
* jrgriffiniii joins08:53
* br2490 joins09:07
* arebenji leaves09:09
* bseeger joins09:11
* acoburn joins09:18
* github-ff joins09:43
[fcrepo4] birkland opened pull request #1081: FCREPO-2093: Remove special case for RDF binary resources (master...FCREPO-2093) https://git.io/vK7Kw
* github-ff leaves
* dchandekstark joins09:58
* dchandekstark leaves09:59
* dchandekstark joins
* dchandekstark leaves10:01
* dchandekstark joins10:05
<apb18>awoods: Is there an "affects version" tag in Jira for 4.6.0-RC2? I couldn't find one, and am in the process of creating Jira issue(s) that could be pertinent to the RC10:09
<awoods>apb18: We could create such a tag... but would 4.5.1 not suffice?10:11
apb18: Then we could set the "fix version" to 4.6.0
<apb18>It suffices for me, though I think one UI anomaly I found only applies to the RC
<awoods>apb18: interesting... and surprising.10:12
apb18: give me a link to your ticket, and I will create the tag.
<apb18>I haven't created the UI one yet, but will let you know once it's in. I just want to verify it in other browsers, and sanity check that it is truly an issue10:13
* dwilcox leaves10:37
* dwilcox joins10:47
* ajs6f joins10:51
* tjohnson joins11:02
* thomz leaves11:16
* whikloj joins11:28
<dchandekstark>awoods ajs6f - jackhill pointed me to comment here yesterday(?) about recommending PostgreSQL b/c of issues with other DBs11:41
are there issues with MySQL other than the one(s) we've reported11:42
<awoods>dchandekstark: I saw ajs6f's message as well in IRC... but no further details.
<ajs6f>awoods: That was entirely in response to your message of yesterday dicsussing Duke's own problems with MariaDB. Seems like PGSQL is the only open source alternative that hasn't demonstrated some kind of problem.11:43
afk bbl
* ajs6f leaves
<dchandekstark>awoods: well, AFAIK no one has replicated our problem(s) with MySQL in other environments11:44
<awoods>dchandekstark: that is my understanding as well.11:45
<dchandekstark>awoods: if there are other red flags w/r/t mysql i'd sure like to know soon
<awoods>dchandekstark: it looks like coblej is seeing the MariaDB issue with MySQL, no?
<jackhill>awoods: we've reproduced the problem with MySQL as well.11:47
<awoods>dchandekstark: He made a comment to that effect: https://jira.duraspace.org/browse/FCREPO-2076?focusedCommentId=51037&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-51037
<jackhill>althought we're seeing the problem with smaller collections with MySQL/MariaDB 5.5 than with later version
<awoods>jackhill: who is "we"?
<jackhill>Duke
<awoods>jackhill: as a data point, I successfully ran the 60k script with: mysql Ver 14.14 Distrib 5.7.13, for Linux (x86_64)11:48
<dchandekstark>awoods: what's your OS?11:49
<awoods>Ubuntu
16.04
<jackhill>dchandekstark: I wounder if we should do a Ubuntu install and try to reproduce there.11:50
<dchandekstark>jackhill: i'm not sure what we would do with that11:51
unless we can get ubuntu with more or less the current support model
<jackhill>I'm thinking it might just be interesting to know11:52
or if it would help us point to a difference that can be fixed.
* jackhill shrugs11:53
* dchandekstark leaves12:07
* dchandekstark joins12:29
* esm_ joins12:32
* bseeger leaves12:42
* bseeger joins12:58
* dchandekstark leaves13:09
* mjgiarlo joins13:28
* mjgiarlo leaves13:57
* jrgriffiniii leaves14:01
* mjgiarlo joins
* dchandekstark joins14:03
* ajs6f joins
* bseeger leaves14:33
<apb18>awoods: no need for a 4.6.0-RC2 jira tag. The UI issue I mentioned this morning turned out to be ... bizarre. It affects 4.5.1+ https://jira.duraspace.org/projects/FCREPO/issues/FCREPO-209414:36
<awoods>thanks, apb18
* bseeger joins
* bseeger leaves14:38
* mjgiarlo leaves14:43
* bseeger joins14:46
<dchandekstark>awoods ajs6f acoburn - one of y'all might be able to save me some time with a weird issue14:48
<ajs6f>dchandekstark: Yeah, I'm pretty weird so I might could help.
<dchandekstark>the REST API POST method on a container sez "If the MIME type corresponds to a supported RDF format or SPARQL-Update, the uploaded content will be parsed as RDF and used to populate the child node properties."
I need to reference the specifics14:49
b/c seeing behavior that suggests text/csv is one of those types
<ajs6f>Wow, did we do that? That sounds kind of useful and cool.14:50
<dchandekstark>it's pretty ugly and awful actually14:51
;)
yes, confirmed
if you use application/csv - goes in as a file
<awoods>dchandekstark: If you want your csv to be a Binary, you can ensure the "Content-Disposition" is included in the request: https://wiki.duraspace.org/display/FEDORA4x/RESTful+HTTP+API+-+Containers#RESTfulHTTPAPI-Containers-post-example6
<ajs6f>What does that look like? Does it take fields from the CSV and try to make RDF out of it?
<dchandekstark>ajs6f: yes, it appears to, no idea how14:52
i went down this road for a completely different reason oddly14:53
a CSV file in the repo that i can't download due apache jena IRI exception
was trying to replicate *that*
with the original file from FCR3 repo14:54
<apb18>dchandekstark: I just submitted a PR for something that sounds suspiciously like it could be related to your issue15:00
<dchandekstark>apb18++
<ajs6f>dchandekstark: Hold on— you couldn't download a binary because of a Jena exception? Or you couldn't download what you _thought_ was a binary but what turned out to have been stored as RDF?15:01
<apb18>let me see, it was due to treating some binary content non-opaquely, and parsing it as rdf. The resource could be created as a binary just fine, but an IRI exception is thrown when trying to retrieve its contents
<dchandekstark>ajs6f: it's definitely a binary - NonRDFSource15:03
<ajs6f>dchandekstark:apb18: apb18 is that your thing that you found?15:04
<apb18>dchandekstark: If Jena thinks its MIME type maps to something it thinks it can parse, it'll try to do so. So I *think* it's the same issue
<ajs6f>Sounds like it's worth dchandekstark trying it.15:05
<dchandekstark>ajs6f apb18 - org.apache.jena.riot.RiotException
IRI resolver barfs on "illegal character in path"
<apb18>ajs6f: Huh, it looks like there's a CSV language in RIOT, so I'm 99% sure it's the same issue that bit me
<ajs6f>dchandekstark:apb18: Yeah, there is a CSV machine in RIOT, but I thought we (Fedora) were not engaging it. I guess we accidentally did.15:06
<dchandekstark>ajs6f apb18 - fwiw, in this specific case, the file is < 4096 bytes, so stored in db (as string?)
<ajs6f>dchandekstark:apb18: There is a whole subsystem for talking about property tables in Jena, but it's not hugely used.
<dchandekstark>coblej just replicated with a larger file15:09
<apb18>ajs6f: This particular case relied entirely on RDFLanguages.contentTypeToLang(contentTypeString) != null and contentTypeString != "text/plain"
<ajs6f>apb18: Wow. Powerful logic, there.15:10
<apb18>yeah...15:11
<dchandekstark>ajs6f: OK, my original suspicion about the mangled download is replicable ...
the default encoding of text/csv according to the RFC is us-ascii15:12
my csv file is utf-8
if i upload it without specifying the charset, then i can't download it from fedora
but if i specify charset=UTF-8 then it works fine
-- that is, with the Content-Disposition header as well15:13
<apb18>dchandekstark: When it 'works', is the file you get back identical to the file you put in?15:14
<dchandekstark>apb18: lemme check that
* esm_ leaves
<dchandekstark>apb18: yes, downloaded file has same SHA115:17
<apb18>dchandekstark: Fascinating! I was able to reproduce the error with a csv file. I'm building Fedora with my PR to see if that fixes it15:20
<ajs6f>acoburn: are you wainting on me to do anything with 209015:24
?
<acoburn>ajs6f: isn't that already taken care of?15:26
ajs6f: looks like that's been merged with both master and 4.6.0-rc
<ajs6f>acoburn: I'm glad I could help.15:27
<acoburn>ajs6f: or were you referring to 1957?15:28
<ajs6f>acoburn: No, I don't think so. Let me ask acoburn.15:29
<dchandekstark>ajs6f apb18 - https://gist.github.com/dchandekstark/192436bf203ed85205f10a9d92f7dd86
<apb18>dchandekstark: OK, so it looks like the PR fixes csv files too. It *is* the same issue15:30
dchandekstark: It's https://github.com/fcrepo4/fcrepo4/pull/1081
<ajs6f>apb18:dchandekstark: Hooray for things that are the same as other things!
<acoburn>dchandekstark: and I believe the plan is to make that fix part of the 4.6.0 release candidate15:31
<apb18>the good thing is that it's only when *reading* the binary that the problem occurs. SO any data already in Fedora is fine.
<acoburn>apb18: isn't that the case ^^^?
<dchandekstark>apb18: too bad people actually want to read the files in the repo ;)15:32
<ajs6f>dchandekstark: One of the problems with Fedora from the beginning has been the community's stubborn insistence on the value of their information.
<apb18>acoburn: Yes, adding the fix to the next RC would be ideal. I just saw your comment about a PR against the RC branch too15:34
<dchandekstark>apb18: so you think "Content-Type: text/csv" triggers the bug, while "Content-Type: text/csv; charset=UTF-8" does not?15:35
b/c that's what i'm seeing
at least with my test file15:36
<ajs6f>dchandekstark: look at apb18's comment above:
RDFLanguages.contentTypeToLang(contentTypeString)
that test might fall through in one case but not the other15:37
* esm_ joins
<apb18>apb18: "Content-Type: text/csv" certainly triggers the bug. I'm surprised "Content-Type: text/csv; charset=UTF-8" doesn't. Maybe it fools RDFLanguages into thinking it isn't an RDF type
<ajs6f>but we'd have to trace it to check
<acoburn>ajs6f: https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/RDFLanguages.html#contentTypeToLang-java.lang.String-15:38
ajs6f: note the phrase (without charset)
* br2490 joins15:39
<apb18>acoburn: bingo. So I'd suspect RDFLanguages is returning null, skipping the special rdf parsing
<ajs6f>acoburn:apb18: right. Look, we def need to make that test smarter.apb18, is that what your PR does?15:40
<apb18>ajs6f: No, it removes the test entirely, and just returns the bytes
<ajs6f>apb18: At a point in execution at which we're already sure it's a NonRDF source?
<apb18>Yes
<ajs6f>apb18: Works or em.15:41
for me
* arebenji leaves15:42
<apb18>ajs6f: It's inaide an 'if (resource() instanceof FedoraBinary)', I'm assuming that's equivalent to "sure it's not an NonRDFSource"
<ajs6f>apb18: Sure, that seems reasonable. I'm willing to believe that.15:44
<apb18>ajs6f, acoburn: Hm, Procedurally, what should I do for a PR to the RC branch. Should I cherry pick the commits onto a branch based off of the RC branch and issue a PR from that?15:50
the issue is that my PR to master is based off master, which contained some commits we don't want in the RC (like re-naming to 4.7.0-SHAPSHOT, etc)
<acoburn>apb18: that's what I did with #1079
<apb18>acoburn: OK, so I'll do that then15:51
* coblej joins15:56
* github-ff joins
[fcrepo4] birkland opened pull request #1082: FCREPO-2093 (Remove special case for RDF binary resources), applied to 4.6.0-RC (4.6.0-RC...FCREPO-2093-RC) https://git.io/vK5K1
* github-ff leaves
* dwilcox leaves16:10
* arebenji leaves16:41
* ajs6f leaves16:44
* acoburn leaves16:45
* apb18 leaves16:47
* whikloj leaves17:00
* mjgiarlo joins17:07
* bseeger leaves17:10
* coblej leaves17:19
* esm_ leaves17:26
* dchandekstark leaves17:32
* dchandekstark joins18:32
* dchandekstark leaves18:37
* dchandekstark joins19:34
* dchandekstark leaves19:40
* github-ff joins19:44
[fcrepo4] awoods closed pull request #1081: FCREPO-2093: Remove special case for RDF binary resources (master...FCREPO-2093) https://git.io/vK7Kw
* github-ff leaves
* dchandekstark joins19:45
* github-ff joins19:46
[fcrepo4] awoods pushed 1 new commit to 4.6.0-RC: https://git.io/vKdtX
fcrepo4/4.6.0-RC 0c3ff77 birkland: FCREPO-2093 (Remove special case for RDF binary resources), applied to 4.6.0-RC (#1082)...
* github-ff leaves
<f4jenkins>Project fcrepo-module-auth-rbacl build #1064: UNSTABLE in 3 min 29 sec: http://jenkins.fcrepo.org/job/fcrepo-module-auth-rbacl/1064/20:02
* travis-ci joins20:05
fcrepo4/fcrepo4#4627 (master - ace587d : birkland): The build passed.
Change view : https://github.com/fcrepo4/fcrepo4/compare/2e76928e449e...ace587d39a26
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/147627737
* travis-ci leaves
* travis-ci joins20:07
fcrepo4/fcrepo4#4628 (4.6.0-RC - 0c3ff77 : birkland): The build passed.
Change view : https://github.com/fcrepo4/fcrepo4/compare/6ec0bf983cdd...0c3ff7740cc6
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/147628063
* travis-ci leaves
<f4jenkins>Yippee, build fixed!20:10
Project fcrepo-module-auth-rbacl build #1065: FIXED in 2 min 51 sec: http://jenkins.fcrepo.org/job/fcrepo-module-auth-rbacl/1065/
* dchandekstark leaves20:12
* dchandekstark joins20:15
* dchandekstark leaves20:16
* dchandekstark joins20:20
* dchandekstark leaves20:44
* mjgiarlo leaves20:45
* dchandekstark joins20:54
* dchandekstark leaves21:50
* dchandekstark joins22:55
* dchandekstark leaves23:01
* github-ff joins23:56
[fcrepo4] awoods pushed 1 new commit to master: https://git.io/vKdRI
fcrepo4/master 67afbf8 Aaron Coburn: Add interfaces for replacement of the existing IdentifierConverter class (#1055)...
* github-ff leaves
* dchandekstark joins23:57
* github-ff joins00:01
[fcrepo4] awoods force-pushed modeshape5 from c177adc to ad6c4a3: https://git.io/vrFht
fcrepo4/modeshape5 6e0b160 Andrew Woods: Upgrade to ModeShape 5.0.0.Final...
fcrepo4/modeshape5 dc185d2 Andrew Woods: Remove references to Infinispan, and default Modeshape configuration...
fcrepo4/modeshape5 13129ff Andrew Woods: Only create one-click if Maven profile is used: -Pone-click...
* github-ff leaves
* dchandekstark leaves00:02
<f4jenkins>Project fcrepo-module-auth-rbacl build #1066: UNSTABLE in 3 min 35 sec: http://jenkins.fcrepo.org/job/fcrepo-module-auth-rbacl/1066/00:13
* travis-ci joins00:14
fcrepo4/fcrepo4#4629 (master - 67afbf8 : Aaron Coburn): The build passed.00:15
Change view : https://github.com/fcrepo4/fcrepo4/compare/ace587d39a26...67afbf83c21a
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/147659382
* travis-ci leaves
* travis-ci joins00:18
fcrepo4/fcrepo4#4630 (modeshape5 - ad6c4a3 : Andrew Woods): The build passed.
Change view : https://github.com/fcrepo4/fcrepo4/compare/c177adc8f9c0...ad6c4a3df254
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/147659965
* travis-ci leaves
<f4jenkins>Yippee, build fixed!00:22
Project fcrepo-module-auth-rbacl build #1067: FIXED in 2 min 51 sec: http://jenkins.fcrepo.org/job/fcrepo-module-auth-rbacl/1067/

Generated by Sualtam