Log of the #fcrepo channel on chat.freenode.net

Using timezone: Eastern Standard Time
* thomz joins02:27
* dchandekstark joins03:06
* dchandekstark leaves03:10
* dchandekstark joins05:07
* dchandekstark leaves05:13
* dchandekstark joins06:24
* dchandekstark leaves06:28
* dwilcox joins07:38
* dwilcox leaves07:39
* dwilcox joins08:34
* dchandekstark joins08:54
* awoods leaves08:56
* manez joins09:02
* bseeger joins09:03
* mikeAtUVa joins09:29
* manez leaves09:34
* ajs6f joins09:39
* peichman joins09:47
* bseeger1 joins10:01
* bseeger1 leaves10:02
* bseeger leaves10:03
* acoburn joins10:06
ajs6f: re fcrepo-2070, I'm somewhat inclined to just ditch fcrepo-transform and run it all in Karaf10:07
ajs6f: configuration inside Spring is going to be a headache for everyone
ajs6f: plus, I'm mostly done with an OSGi impl
ajs6f: and even more, the Sparql portion of fcrepo-transform doesn't work anyway10:08
* dwilcox leaves10:10
* dchandekstark leaves10:11
* dchandekstark joins10:12
* dchandekstark leaves10:18
* dwilcox joins10:23
* dchandekstark joins
* bseeger joins10:24
* dchandekstark leaves10:25
* bseeger leaves
* dchandekstark joins10:26
* bseeger joins10:27
* dchandekstark leaves10:34
* dchandekstark joins10:36
* dchandekstark leaves
* dchandekstark joins
* peichman leaves10:38
* dchandekstark leaves10:42
* dchandekstark joins
* dchandekstark leaves10:46
<barmintor>I am really in identifier conversion hell here10:53
types of identifiers in fcr:
* manez_ joins
<barmintor>1) The externalized URI10:54
2) the path fragment of an externalized URI
3) a processed path fragment with translations of hashes and namespaces10:55
4) an internal identifier that resolves fcr:versions and the signalling suffixes (fcr:metadata, fcr:tombstone)
5) a JCR path10:56
this is supposed to be a chain of inverse functions
but also, at any point, might feed an externalized URI that has not been subject to 2 & 3 inversions back into the top10:58
* bseeger leaves11:00
* dchandekstark joins11:02
* bseeger joins
* dchandekstark leaves11:07
* dchandekstark joins11:08
* thomz leaves11:14
* dchandekstark leaves11:19
* dchandekstark joins11:23
* acoburn1 joins11:24
* acoburn leaves11:25
* jjtuttle leaves11:26
* jjtuttle joins11:27
<ajs6f>acoburn: How fast could we have a Karaf-side impl11:30
<acoburn1>ajs6f: probably this week
ajs6f: that is, I have the code written; I need to just add integration tests and work out deployment11:31
<ajs6f>acoburn1: I say, let's do it. We can put out the fresh functionality as we have it, and start deprecating fcrepoi-transform in 4.7
* jjtuttle leaves11:34
* jjtuttle joins11:35
* jjtuttle leaves
* jjtuttle joins11:37
<dchandekstark>y'all must be ready to hear the latest weird problem we're experiencing at duke11:38
<barmintor>let's hear it!11:39
<ajs6f>Random refrigerator fires?
<dchandekstark>barmintor: hey
<barmintor>dchandekstark: hey hey
<dchandekstark>ajs6f: actually that's not far off
<ajs6f>Random refrigerator hurricanes?
<barmintor>I hope that means the refrigerator part is right and not the random
<dchandekstark>random but fairly numerous non-reproducible fixity check failures
<ajs6f>It's okay, barmintor is working on the fixity spec. He just needs to add in a section requiring exactly that behavior.11:40
<barmintor>dchandekstark: I'm assuming this is against 4.5.2-SNAPSHOT11:41
* jjtuttle leaves
<dchandekstark>barmintor: no, i think it's actually just 4.5.1
dchandekstark: setting aside the reproducibility, what are the circumstances of the check? through hydra? retrieved and validated against checksum stored?11:42
<dchandekstark>barmintor: thru hydra, yes, but i monkey patched the FixityService so I could dump the raw response from fedora11:43
and the BAD_CHECKSUM string is there
so i don't think it's hydra's fault
the fedora code doesn't log anything when there's a bad checksum11:44
it just logs that the fixity service is called
barmintor: oddly, spot checking one failure, the reported size is correct11:45
but the message digest is wrong
barmintor: FedoraBinaryImpl has a debug message that we'er not getting altho i'm not sure it would be useful11:47
the fact that it never seems to fail on retry smells like a cache issue?11:48
* mikeAtUVa has observed that the number of times a failed fixity check identifies a problem in software or process is infinitely higher than the number of times failed fixity checks indicate a data file corruption.
<barmintor>and the errors are consistent on a given resource?
<dchandekstark>barmintor: no, non-reproducible so far11:49
<barmintor>dchandekstark: that is, a resource failing a check keeps failing a check, or it fails, then passes, etc.
<dchandekstark>a resource failing a check once then passes on every subsequent check
<barmintor>dchandekstark: ok, that definitely sounds like a cahcing issue11:50
<dchandekstark>barmintor: esp looking at https://github.com/fcrepo4/fcrepo4/blob/master/fcrepo-kernel-modeshape/src/main/java/org/fcrepo/kernel/modeshape/FedoraBinaryImpl.java#L294
which is the same in v4.5.111:51
<barmintor>something something 2 hard problems11:52
<ajs6f>dchandekstark: Keep in mind that INSP refers to entires as Cache Entries because it thinks of itself as a cache, even tho MODE and we are using it as a persistent store.
<barmintor>ajs6f: I think this language is our side, but I also had that thought11:53
* dchandekstark leaves
<ajs6f>barmintor: We took over their language somwwhat, I think.
* dchandekstark joins11:57
<barmintor>oh, fun. I don't have a ISPN impl anymore.
<dchandekstark>barmintor: 2 hard - cache invalidation and naming things
<barmintor>dchandekstark: yeah
<dchandekstark>barmintor: so, i can retry once on a failure, although that makes me a bit squeamish11:58
<barmintor>dchandekstark: yeah, I'm not happy to recommend that
dchandekstark: let me pull master and see if there's anything we can do on the mode4 side11:59
<dchandekstark>barmintor: case in hand is replicating to duracloud, so i'd kinda like to know beforehand that the resource is good :)
<barmintor>dchandekstark: agreed, I just need to see if there's a way to tell MODE4 to firce the issue with ISPN12:00
* jjtuttle joins12:01
* dchandekstark leaves12:04
* ajs6f leaves
* bseeger leaves12:10
* dchandekstark joins
* bseeger joins
* bseeger leaves12:11
* bseeger joins12:12
* bseeger leaves12:14
* dchandekstark leaves
* dchandekstark joins12:16
* bseeger joins12:19
* dchandekstark leaves
* jjtuttle leaves12:22
* jjtuttle joins12:24
* dchandekstark joins12:25
* peichman joins12:27
* dchandekstark leaves12:30
* bseeger leaves12:31
* dchandekstark joins12:33
* jjtuttle leaves12:37
* dchandekstark leaves12:38
* dchandekstark joins12:41
* jjtuttle joins12:42
* dchandekstark leaves12:45
* dhlamb joins12:55
* arebenji joins
* jjtuttle leaves12:59
* peichman leaves13:00
* jjtuttle joins
* jjtuttle leaves13:02
* whikloj joins13:03
* jjtuttle joins
* jjtuttle leaves13:09
* acoburn1 leaves13:17
* acoburn joins13:18
* bseeger joins13:34
* jjtuttle joins13:41
* jjtuttle leaves
* whikloj leaves13:57
* jjtuttle joins14:03
* dchandekstark joins14:26
* ajs6f joins14:29
ruebot: How does "BagIt-LDP" address the multiresource bag questions?14:30
* dchandekstark leaves14:35
* dchandekstark joins
<ruebot>ajs6f: ttl in the bag tags...14:40
<ajs6f>ruebot: That's not really the issue. It's not the serialization, it's the API.
<ruebot>ajs6f: ah. well...14:41
* ruebot thinks
<ajs6f>ruebot: Thanks, tho. ruebot, always the peacemaker.
ajs6f: https://images8.alphacoders.com/419/419907.jpg only know peace, love, and happiness :-D14:42
<ajs6f>ruebot: http://bit.ly/29YH3rC14:43
* ruebot grins
* github-ff joins14:46
[fcrepo4] ajs6f closed pull request #1085: Upgrade to Modeshape5 (master...modeshape5) https://git.io/v6exw
* github-ff leaves
* dhlamb leaves14:51
* dchandekstark leaves14:52
* dchandekstark joins14:59
* dchandekstark leaves
* dchandekstark joins
<f4jenkins>Project fcrepo-transform build #221: FAILURE in 5.8 sec: http://jenkins.fcrepo.org/job/fcrepo-transform/221/15:00
Project fcrepo-module-auth-rbacl build #1070: FAILURE in 3.1 sec: http://jenkins.fcrepo.org/job/fcrepo-module-auth-rbacl/1070/
<dchandekstark>barmintor: any light on the fixity issue? just curious ...
<barmintor>dchandekstark: I can see where it would be a problem with ISPN clustered, but I don't see a way to just fix it. It's possible that monkeying with the15:03
eviction strategies would fix it15:04
but I'm surprised that it's even an issue in a single-node configuration
<dchandekstark>barmintor: i've wondered about eviction in general
but don't really have a handle on exactly how it affects various things15:05
<f4jenkins>Project fcrepo-webapp-plus build #718: UNSTABLE in 1 min 17 sec: http://jenkins.fcrepo.org/job/fcrepo-webapp-plus/718/
* acoburn leaves
<barmintor>dchandekstark: of course it all goes away in MODE5, so ...
dchandekstark: b/c I think this kind of thing is one of MODE's frustrations w/ ISPN
<dchandekstark>barmintor: yes, well, we're on migration 2 of ? on FCR 4.5.115:06
<barmintor>I know, I know
<barmintor>dchandekstark: I promise, you're not alone in being frustrated with the MODE/ISPN situation.
<dchandekstark>barmintor: i believe you ...
we're just in a little bit of a pickle15:07
* travis-ci joins
fcrepo4/fcrepo4#4639 (master - ad94239 : Aaron Coburn): The build passed.
Change view : https://github.com/fcrepo4/fcrepo4/compare/67afbf83c21a...ad9423914a9d
Build details : https://travis-ci.org/fcrepo4/fcrepo4/builds/149555611
* travis-ci leaves
<dchandekstark>b/c want to move forward with F4 ASAP
but feels like the version we want to be on is always the next one, or the one after that :(15:08
barmintor: fwiw, i may have persuaded myself that retrying the fixity check *once* is ok in the short term15:09
<ajs6f>dchandekstark: if it makes you feel any better, mikeAtUVa feels the same way, and UVa helped _write_ Fedora 4.15:10
<dchandekstark>ajs6f: somehow that doesn't make me feel better ...15:11
ajs6f: thx for trying though15:13
<ajs6f>dchandekstark: If it makes you feel any better, I'm not really very good at making people feel any better.15:14
<dchandekstark>ajs6f: hey, that worked a little ;)
ajs6f: fwiw, neither am i
i'm kind of the gloomy gus around here :)15:15
<ajs6f>dchandekstark: If you are on the front lines with Fedora, you have the right to be as gloomy as you want.
* bseeger leaves
* bseeger joins15:17
<barmintor>well, I try to think of it this way15:30
Fedora 3 was in large measure a codebase continued from Fedora 2 (you saw it in lots of the source)15:31
<ajs6f>Much of the XML procesing was written by Sandy herself.
<barmintor>which is to say that it took the community a decade to build it and identify its problems
(and the community's limits)15:32
Fedora 4 is attempting to circumvent the latter by cutting about 200k lines of code out of the codebase
but we're not very far into the former15:33
<ajs6f>Hm. Fedora 4 (IMNSHO) is trying to circumvent the latter by moving from producing software to producing standards.
<barmintor>I think these are of a piece
<ajs6f>Well, I guess if you cut _all_ of the lines of code, you are left with my view as the limiting case.15:34
<barmintor>but the sponsors of the FF project regarded building on external bases as a virtue (and so did we)
<ajs6f>did and do, for code and standards15:35
<barmintor>but there was always the possibility too that some of the things that were hard were actually hard
and I think seeing JBoss work through these issues points at that15:36
<ajs6f>That's why I want to stop writing software and leave that to others.
<ajs6f>I don't think Duke's woe is because of the difficulty of the problems in hand. It's because of the use of the words "production ready" to advertise the software product.15:37
Whatever those words mean.15:38
In some ways, the problem with those words is that everyone thinks they mean "At least as useful and low-maintenance as my current production solution."15:39
Which is pretty much never true, as barmintor points out.
<barmintor>I would protest both of those, as I find myself (even this very FCR4 intensive week) doing orders of magnitude less maintenance on FCR4 than I did on 315:40
<dchandekstark>ajs6f: so the purpose of F4 is not to produce something useful, but to define how something should work, if it exists? ;)
<ajs6f>barmintor: Were you on the tech call during which dchandekstark talked about Duke's emotions?
<barmintor>No, I was in Spain15:41
<ajs6f>barmintor: Yes, I'm sure you do. But that has more to do with your long-suffering and gentle nature than the software in hand.
dchandekstark: That's where we're going, yep. And what's more, we're going to make that definition so simple and obvious that your first reaction will be, "But I could just do that myself!"
<barmintor>ajs6f: maybe, I only want to observe that FCR3's maintenance load was an illusion
<ajs6f>dchandekstark: And you will.15:42
barmintor: Meaning it had been amortized across a decade?
<barmintor>ajs6f: and then piled high on 1 institution at the end
both of which hid the overall cost15:43
<dchandekstark>ajs6f: but isn't the idea of the fedora project at least in part that i don't ahve to do it myself (even if i could)?
<ajs6f>barmintor: The first part, true. The second… well, it's not like Rob ever said, "Hey, we can't do this any more!"
<barmintor>ajs6f: Rob Cartolano?
ajs6f: he literally said that
<ajs6f>barmintor: BUt did he actually tell you to stop?15:44
barmintor: Before starting F4?
<barmintor>ajs6f: yes!
<ajs6f>barmintor: Then why didn't you stop?
<dchandekstark>ajs6f: thank god he didn't stop :)15:45
<ajs6f>dchandekstark: Yes, and that idea shouldn't go away. It's just that we'll be doing diffferent things together, and there will be new groups working as well. E.g. acoburn has been interested for a while in high-volume work based on Apache Spark, and that might or might not interest you. barmintor (and I) are interested in a super-simple triplestore + filesystem arrangement, and that might be up your alley. You'll choose who to work with from15:46
<barmintor>ajs6f: because I wanted things to be better. FCR3 was my hobby.
a horrible hobby.
<ajs6f>barmintor: I think that sums it up pretty well.
<barmintor>ajs6f: yeah, but that is completely untenable as a strategy.
<ajs6f>dchandekstark: But when it comes to, say, indexing, or other kinds of things one does with a repo (as opposed to what the repo does itself) the community will keep on keeping on.15:47
<dchandekstark>barmintor: which?
<barmintor>dchandekstark: having a project depend on some random person dedicate an hour or so every morning before they go to work
<dchandekstark>ajs6f: ok, but why don't we just say we're doing microservices and be done with it
<ajs6f>dchandekstark: I have no idea what you mean.15:48
* dwilcox leaves
<ajs6f>barmintor: It is, it certainly is (untenable).
<dchandekstark>ajs6f: i guess i just missed the boat on what Fedora 4 is supposed to be15:49
<ajs6f>dchandekstark: I dunno: it's changed a good deal as we've learnt and grown.
dchandekstark: It certainly is not just "Fedora 3, but not as crappy and old."
* dwilcox joins15:50
<dchandekstark>ajs6f: i realize the insiders see all the warts in F3
but it's been damn reliable for us
<ajs6f>dchandekstark: Not just insiders. Plenty of people who have never seen the inside. And if you want to maintain it, rock on. That's not intended as snarky, I'm perfectly serious. But don't expect anyone else to do that work for you.15:51
dchandekstark: The Fedora Futures effort started because it was becoming clear (it is now completely clear) that the Fedora 3 code base was unmaintainable (barmintor or no barmintor). And it is not really feasible to do anything new with it. (add features.)15:52
<dchandekstark>ajs6f: cool, fine.15:53
<ajs6f>What is the sound of one barmintor maintaining?
* ajs6f wails like a banshee
<dchandekstark>but it seems like fedora as preservation platform was abandoned for fedora as LDP15:54
i'm still not sure why the project wasn't renamed
<no_barmintor>the sponsors didn't want to rename it15:55
<ajs6f>dchandekstark: Hm. Those seem like pretty orthogonal concerns to me, but that would depend on what you mean by "preservation platform" and just how different your definition is from everyone elses.
BECAUSE IT IS DIFFERENT. After fifteen years of listening to people talk about this, I guarantee it.
<no_barmintor>I also wouldn't muddle the LDP part, which I think is more preservation friendly in a way, with the MODE part.
<ajs6f>Yes, the MODE thing is a case of "We always do the right thing with the info and wisdom we have, at a given time. Sometimes, we get new info or get wiser and we realize that we can make a better choice."15:56
dchandekstark: DId you come to OR in Dublin?
<no_barmintor>dchandekstark: but we were responding to an economy of sponsor use cases, including clustering, which didn't feel we could deliver w/o trusting a vendor
<dchandekstark>ajs6f: no15:57
<ajs6f>And we got a lot from MODE. We got a massive jump start.
<dchandekstark>ajs6f: i will be happy to become enlightened in time to the wisdom of the current direction ...15:59
<ajs6f>dchandekstark: I did a presentation with awoods where we (obliquely) talked about this. I claimed that Fedora is a set of ideas and a community, which is interested in using those ideas to create durability in networked information. The community builds software (and now standards) together to realize those ideas. From that POV, the continuity is perfect obvious. If you think of Fedora as primarily a software project that we must keep roll
dchandekstark: That's not to dismiss your concerns. It's just to try to give some insight into why we have done what we have done.16:00
Other than booze. Booze probably had a lot to do with it.16:01
<dchandekstark>ajs6f: i get that, i do, and i'm not saying it's bad
<ajs6f>Booze? No, booze is not bad at all.
<dchandekstark>i'm just still feeling that there's a pretty big disconnect between a community of *users* that are invested in Fedora-based repository infrastructure
and the F4 start-over approach16:02
listen to smithsonian (I think) guy on the import/export call if you were there
<ajs6f>dchandekstark: I consult for SI— I know Dan Davis well.16:03
dchandekstark: They were part of the Fedora Futures efforts.
dchandekstark: (and I was on the im/ex call)
<dchandekstark>When he was talking I kept thinking - you just want F3 :)
<ajs6f>dchandekstark: No, he wants something that actually scales to the levels he needs and works with the linked data resources his researchers are bringing to him. Trust me. I work with him daily.16:04
<dchandekstark>ajs6f: i'm just saying, ideas and communities are important, but so are *products*
ajs6f: ^ i just meant w/r/t import/export, but ok, ok
<ajs6f>dchandekstark: They sure are. The question is: is the purpose of the Fedora community to build a product. I would say, no, and it never was.
It's just something that we happened to do. Other groups may be just as good at doing that, and a hell of a lot more agile.16:05
dchandekstark: Part of what's at stake here is the question of whether a single centralized softwre effort actually _can_ do all of the things that the Fedora community has asked of Fedora.16:06
dchandekstark: That is not a question with an obvious "yes" answer.
<dchandekstark>ajs6f: does it *have* to do *all*?
<ajs6f>dchandekstark: That's exactly the point of the move to standards. If there is only one product, then _YES_, either it does it all, or someone is disappointed. And many of the demands heard in the community are at odds with each other at the technical level.16:07
dchandekstark: In a Fedora world of multiple impls, you choose the one that has the characteristics that suit you.
dchandekstark: For example, Duke might be interested in simplicity and reliability, but Amherst might choose for advanced stream processing capabilities.16:08
dchandekstark: acoburn's folks want to use Apache Spark on their data, but they also want him to manage it in sond ways. I have no impression that anyone is asking you for Apache Spark workflows. Why would you want to pay for them?16:09
dchandekstark: I suspect that if you had to hand a really simple, utterly reliable and transparent impl of the API specs, you would feel a lot better about using Fedora into the future. Right now, you have only the choice of the (much smaller than F3 but still) large and complex MODE impl. We're going to fix that.16:12
* arebenji leaves16:14
* peichman joins16:22
* dchandekstark leaves16:24
<ajs6f>dchandekstark: I think I probably haven't convinced you of much of anything :) But again, I'm not trying to dismiss your concerns. I encourage you to talk to jjtuttle and dwilcox, especially. Much of the messaging out of the tech team on these topics is aimed at the Fedora Leadership group. It sounds to some extent like you are saying that it may not be getting as far beyond that group as it should (with respect to these matters of vision a16:25
* manez leaves
* peichman leaves16:31
* dchandekstark joins16:45
ajs6f: ^ that makes me feel better, sorry i was away for a while16:46
<ajs6f>dchandekstark: NP. This is important stuff and I don't want you to feel like Fedora is puking on your shoes. Well, after the versioning bug, you probably feel like that already, but let's try to minimize the quantity. Duke has been way out in front taking hits for the community and we're thankful.16:49
* dwilcox leaves16:50
* dhlamb joins
<ruebot>dchandekstark++ ajs6f++
* dchandekstark leaves16:56
* ajs6f leaves16:59
* dchandekstark joins17:02
* bseeger leaves17:05
* mikeAtUVa leaves17:06
* dchandekstark leaves17:07
* dchandekstark joins17:08
* dchandekstark leaves17:12
* dhlamb leaves17:16
* peichman joins17:35
* peichman leaves17:37
* dchandekstark joins17:39
* dwilcox joins17:41
* dchandekstark leaves17:43
* dchandekstark joins17:44
* dwilcox leaves17:47
* dchandekstark leaves17:54
* peichman joins18:30
* peichman leaves18:34
* dwilcox joins18:39
* dwilcox leaves19:06
* dchandekstark joins20:19
* dhlamb joins21:34
* dhlamb leaves22:49
* dhlamb joins22:52
* dchandekstark leaves23:11
* dchandekstark joins23:25
* dchandekstark leaves23:28
* dhlamb leaves00:05

Generated by Sualtam