Log of the #duraspace-ff channel on chat.freenode.net

Using timezone: Eastern Standard Time
* eddies leaves00:43
<cbeer>git++00:58
* eddies joins01:14
* eddies leaves
* eddies joins
* eddies leaves01:23
* eddies joins01:32
* eddies leaves
* eddies joins
* eddies leaves04:58
* anusha joins05:15
* eddies joins07:44
* eddies leaves
* eddies joins
anusha: does https://gist.github.com/raw/cc2c34642d17bff401a1/cc09c4d1139045276c33c00c7b99cafcca698d13/jmeter-fedora-268435456mean-282563637stddev-1threads-bymethod.png show that a 1200byte request took between 10 and 100 *seconds* to complete?!07:48
* jonathangee joins08:10
* JasonDGI joins08:17
* fasseg leaves08:25
<anusha>eddies: yes.08:58
the link to the corresponding csv file is https://gist.github.com/raw/cc2c34642d17bff401a1/1a92f778b30793030ae52c881f85f540a907129d/jmeter-fedora-268435456mean-282563637stddev-1threads.csv08:59
<eddies>those numbers don't make a lot of sense to me09:00
it doesn't take 10-100seconds to post a 1.2k blob to fedora09:01
<anusha>exactly, that's why I have been staring at the test and looking at the test plan again. It's the same test plan as before.09:04
and it also explains why my tests take so ling to finish
I had a lot of trouble getting fedora to work yesterday evening (it started giving 404s and 500s in the middle of a test run) and one of the guys (Chris, Jason) seem to have fixed it.09:05
<JasonDGI>chris++ on that one09:06
<anusha>Thanks JasonDGI. Good to know. I need to find out from Chris what went wrong then.
<eddies>anusha: can you setup/test fedora on another one of the vms as a sanity check?09:11
<anusha>eddies: I will do that right away09:12
<eddies>ty
<anusha>eddies: Can I do that on futures 1 or do I need to do this on a new VM09:13
I do not want to touch 2 & 3, as I have feeling spring is the reason for this
<eddies>doesn't really matter as a sanity check09:28
* fasseg joins09:42
<anusha>eddies: On my dev machine, the numbers are looking a lot better. I will plot these also10:07
<barmintor>20 signed up for Monday; still 5 for Tuesday10:28
<eddies>19 actually for monday, most recent was a test entry by me. deleted now10:37
<cbeer>i noticed trouble too, but didn't get anywhere last night.
i tried on futures5 too, same deal.. so today i'll try locally and see if it was something we've done (eventing? spring? etc), or what.10:38
<anusha>The test results from a test run on a dev machine https://gist.github.com/9e6061009eaf752f81fd10:39
<cbeer>anusha: where are you getting your data for bytes?10:40
(the bytes column in that csv file is the response size in bytes, not the request size)
<anusha>In the test plan you wrote, I think you generate them just before adding them
<cbeer>ok
<barmintor>eddies: I reserved a car for Tuesday morning, to return /wednesday night
<eddies>just reserved (as in not prepaid and can still cancel)?10:41
<barmintor>eddies: yes
<eddies>cool
i'm debating if we're going to need a car for longer
<anusha>cbeer: Oh, ok. that explains why the size is always in two bins
<eddies>i can just imagine us having to run out and get stuff on mon & tues too
so i'm looking at weekly rates10:42
<barmintor>eddies: ok. I just wanted to make sure we were on the books, as it were.
let me know
<eddies>you don't already have car insurance, do you?
<anusha>I'll modify the test to log the generated bytes
<barmintor>no, not anymore
<eddies>as in you personally because you own a car and etc.
ok
<cbeer>yeah, i'm still seeing as good performance as ever locally. i'll make sure we have the same versions of stuff, etc.10:46
<anusha>cbeer: Shouldn't the response size for a binary read reflect the data size?10:55
it's about a 100 times slower in futures6
<cbeer>anusha: it would, except ajs6f caught a problem there and looks like i didn't fix it.10:56
there should be a /content at the end of the Path
<anusha>oh yes, ofcourse!
* ajs6f joins11:02
<cbeer>i'll be on in a sec
<ajs6f>I'm on another conference call, but I'll hang out in IRC for any questions.
<anusha>It's gone silent on the call for me again.11:03
<eddies>i'm muted
jasondgi can you add the pleasing dns name as a comment to the ticket too?11:05
<cbeer>https://www.pivotaltracker.com/story/show/43486027
(and, i'd note, this is a NEW issue on futures6)11:07
<eddies>anusha: actually, carol isn't going to be ready for 20mins, so let's talk now11:13
<cbeer>short_standups++
<anusha>okay11:14
my skype id anusha.rang11:15
<JasonDGI>standup_duration--11:16
<anusha>eddies: was the graph anything like what you were looking for?
<cbeer>ok, that was esay. i forgot about a configuration change that fixes the infinispan stuff
<eddies>anusha: can you confirm my skype contact request? =)
<anusha>eddies: done11:17
* github-ff joins
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/7ksh8g
ff-modeshape-prototype/master edbad8e Chris Beer: add perWrite file sync property to fix our file handle issue, I hope
* github-ff leaves
<cbeer>ok, now then. i'm going to fix up the jgroups config, then look at the VMs. I assume if I turn up anything that requires a sysadmin, nothing will happen until next week, right anusha?11:18
JasonDGI: i don't think we want to commit the jmeter logs, do we? re: "Removed gitignore from logs so we can commit logs"11:19
or did i miss a decision sometime?
wait, now i'm all confused
<JasonDGI>so am i
<cbeer>Jason MacWilliams <anusha.ranganathan@ieee.org>11:20
one of you has your git name set wrong
<JasonDGI>i dont think thats me
futures5?
<cbeer>in any case, i'm on the record as thinking mixing data and code is a bad idea.
<JasonDGI>git config --global --list11:22
user.name=Jason MacWilliams
user.email=jason@discoverygarden.ca
<cbeer>hm. i should hook up the ff-jmeter-madness post commit hook
<JasonDGI>its not me
* github-ff joins11:23
[ff-jmeter-madness] none pushed 3 new commits to master: http://git.io/uRvtRg
ff-jmeter-madness/master 0396b6e Chris Beer: fix test_label property
ff-jmeter-madness/master a7c1a07 Jason MacWilliams: Removed gitignore from logs so we can commit logs
ff-jmeter-madness/master 14e9184 Chris Beer: fix read binary request path
* github-ff leaves
<cbeer>there.
so, i guess that leaves me with just that VM problem
JasonDGI: are you using fedora on futures6 now? (i'm happy to go test on a different VM)11:24
JasonDGI: i forget, there's fixture objects of some sort islandora needs, right?11:27
s/fixture/default/
and if i blow away fedora, i should recreate them11:28
<JasonDGI>i am using it
and im just about to restart it to turn on the resource index
<cbeer>ok. i see there's 60G of data in there
and we've used 91% of the disk on that machine
<JasonDGI>did you need to nuke it?
<cbeer>presumably left over from anusha's testing
huh. i'm not seeing the modeshape slowness i saw before11:29
hold on, let me try fedora again too
<JasonDGI>let me know when you're done, cause i need to cycle tomcat
<cbeer>nope, all speedy.
<JasonDGI>unless you are going to reinstall the hope thing
*whole thing
<cbeer>wasn't my plan. i just wanted to clear out old data. but, carry on. i might do something after you're done for the day.11:30
oh, speaking of things to set up.. do you know much about gsearch config?
<JasonDGI>kk, shutting it down
<cbeer>anusha: yeah, i'm stumped. i can't recreate the problems i was seeing any more.11:32
<JasonDGI>ok, ts back online
<cbeer>i'm going to pretend it has something to do with the disk resizing, until proven otherwise.. like when they resized it, they had to physically move our VM across disks
and, since it was doing that on-line, it needed to throttle us, or do double-writes, or a transaction log.. or something.11:33
(but i don't know anything about that kind of thing, so...)
ajs6f: the TRACE level logs from modeshape are very nice. i should have started there when i was looking at how this is all wired together11:34
<ajs6f>Cool— I've never turned that on. Maybe I should have done that a while ago. {grin}
<cbeer>ok, and now, jgroups 3.2.. and i might actually go into the office before noon today.11:39
* github-ff joins11:40
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/aNokzg
ff-modeshape-prototype/master f529a81 Chris Beer: bump infinispan config to the 5.2 schema
* github-ff leaves
<cbeer>and i'm going to try to ship a jgroups config that uses TCPPING instead of UDP multicast or JDBC_PING11:43
to silence those network unreachable errors
<anusha>cbeer++11:45
cbeer: I'll run tests over the weekend and monitor.
<cbeer>thanks anusha11:46
ajs6f: do you know if java properties can be injected into the modeshape repository json? i'm having a hard time coming up with the right words to search for11:47
<ajs6f>Yes, give me a second to find it.11:48
<cbeer>i'm going to try to make the jgroups config configurable by a java property
<ajs6f>https://docs.jboss.org/author/display/MODE/ModeShape+in+Java+applications#ModeShapeinJavaapplications-Variables11:49
<cbeer>ajs6f++
<ajs6f>+1 to that. Configurability is good. {grin}
<cbeer>oh, that variable expansion is obvious. guess i could have just tried it11:52
i'm also going to cluster modeshape instances too11:54
ajs6f: i haven't tried the 10/100 test yet. let me fire that off now11:59
against futures512:00
<ajs6f>Rocking.
* github-ff joins
[ff-modeshape-prototype] cbeer pushed 1 new commit to master: http://git.io/p80o2Q
ff-modeshape-prototype/master ab77635 Chris Beer: update to jgroups 3.2 configuration, use TCPPING for cluster pinging. configured modeshape-level clustering too. added java properties for using different jgroup configurations.
* github-ff leaves
<ajs6f>cbeer- what do you mean by clstering modeshape?
as oposed to INSPN?
<cbeer> "clustering" : {
"clusterName" : "modeshape",
"channelConfiguration" : "${fcrepo.modeshape.jgroups.configuration:jgroups-modeshape.xml}"
},
in addition to ISPN
<ajs6f>Hm. Do we know what kind of state is thereby being shared?12:02
Is JCR Session related?
Or config?
<cbeer>i think it's session-related.12:03
i found it in the example configs and some pretty diagrams
let me try to find that page again
ok, running 10t 100x test now12:08
and i'm not finding the right page
i'll look again later and be sure to send it to the list this time
and we're consistently holding at 1177 file handles12:09
1/5 through the test
afk
<anusha>cbeer, ajs64: Sorry to break into your conversation. Do you know if it's even possible to save the file size of post in the graph results?12:17
in jmeter. Or do I need to save this to a log file and then run a separate script to add it to the csv file12:18
<JasonDGI>so islandora is up and running (http://islandora.fedora4lib.org/) against fedora, is the modeshape api ready for me to test?12:24
* anusha leaves12:35
<cbeer>JasonDGI: i don12:37
t think it's running on futures6 right now
<JasonDGI>kk
<cbeer>ajs6f: 10t 100x ran, file handles held steady12:38
<ajs6f>By the power vested in me as requestor of the issue, I now pronounce this issue CLOSED!
And devliered.
<cbeer>JasonDGI: i was going to start it for you, but that's silly:12:39
cbeer@futures6:/opt/ff-modeshape-prototype$ mvn -Djetty.port=9999 clean jetty:run
do that in that directory.
and the modeshape/fedora api will be at localhost:9999/rest/
<ajs6f>Keep in mind that you might want to 'nohup' it if you intend to leave it running.
<cbeer>ajs6f: at some point, we can probably compile it as a war and drop it into tomcat, right?12:40
<ajs6f>We can do that now.
<JasonDGI>is it ready on futures6, or should i run it somewhere else?
<ajs6f>mvn clean package
<cbeer>JasonDGI: futures6 is fine
ajs6f: cool. i might do that at the end of the day
<ajs6f>Right on.12:41
<cbeer>before code4lib at least
<ajs6f>mvn jetty:run is handy for development, but if you know you're going to leave it running...
why not set up a container once and have done with it.
<cbeer>ok, heading to the office.12:43
<eddies>barmintor: your registration counter seems to be slightly borked12:53
<barmintor>eddies: I don't think the summary formulas are updating when rows are inserted :(12:54
<eddies>heh12:55
oh for the rental car, were you able to arrange a return location near the house?
you don't have to haul back to o'hare, right?
<barmintor>oh, I didn't even think of that.
I should change that reservation.
<eddies>=)
<cbeer>ok, so i guess i should try to deploy our thing under tomcat now13:12
ajs6f: is there anything worth doing to prepare for modularization?13:13
or is it an all-or-nothing kind of thing?
<ajs6f>I think the most important thing to come to a really strong understanding of what will and will not be int the core.
E.g. FOXML… out.
<cbeer>ok. that sounds like a good face-to-face discussion to me13:14
<ajs6f>Eventing… internal in.
JMS/RSS/external out.
<cbeer>maybe we should throw ideas onto the wiki?
<ajs6f>Sure.
<cbeer>s/ideas/ideas of face-to-face topics/
<ajs6f>Eddie— shall I put a scratch page near the planning pages for c4l?
<cbeer>or.. deliverables from our first week? i don't know how to frame this in the most productive way13:15
<ajs6f>Ah, that would be better in some ways. So for modularization: "A commons understanding of the functionalities allotted to the kernel, and an agreed-upon choice for modularization framework." And you know which one I'm rooting for.13:16
https://wiki.duraspace.org/display/FF/Special+deliverables+from+c4l13:20
I dunno, Chris. If we have no substantial disagreement right now about what goes into the kernel and what doesn't, and people are willing to buy OSGi at least to start with, we could do this beforehand and come to c4l in a better position to help other people hack on the codebase.13:21
I'm going to do a little research on ModeShape in OSGi. There might be pitfalls there (crosses fingers).13:30
<cbeer>I'm totally willing to buy OSGi
if you say it works
<ajs6f>Cool. I'll report back with what I find (Hopefully "ModeShape loves OSGi!"
<cbeer>I feel like we had some agreement about the kernel after CNI, but probably best to sketch it out again face-to-facet.. and actually assign module names to them this time13:33
that DefaultFilter is slick. ajs6f++13:37
<ajs6f>Eh?
<cbeer>finally getting around to seeing what you did with the event stuff13:38
<ajs6f>Oh, as long as it makes sense. I'm not really much of a Java programmer. I keep trying to escape into functional idioms.13:39
You know, from a certain weird point of view, you could think of constructor-injection as a way of introducing functional idiom into Java.13:42
<cbeer>i'm going to make some epics for the high-priority items we identified after CNI13:48
<ajs6f>Well, all I found was that MOdeShape jars come with osgi metadata, so that's nice. As JBoss AS is an OSGI container, so presumably that metadata works. So the next step is to do some simple testing. I'm going to take an empty Karaf (my favorite OSGi container) and try to put ModeSHape and its own HTTP API into it. If it works, I'll look at our codebase. If it doesn't not much time lost.13:49
afk13:50
<cbeer>ajs6f: i'd be very interested in walking through the modeshape/infinispan persistence stuff next week. i looked at some of the stuff you mentioned yesterday, but i feel like i'm still missing pieces (or, have too many pieces)14:15
* jonathangee leaves14:33
<ajs6f>Stuff I mentioedn yesterday/ You mean in the INSPAN and MODES code?14:34
* jonathangee joins14:35
<cbeer>yes.14:40
<ajs6f>Okay. I'm still hopeful that we have what we want here, but I could easily be wrong.14:41
<cbeer>hm. what to do next.14:45
oh, deploying to tomcat.14:46
<ajs6f>I'm about to add an issue for OSGi-ification to Pivotal. You can take a lok at that and tell me where I went wrong. {grin}
https://www.pivotaltracker.com/story/show/4367547514:48
<cbeer>it might be good to do a osgi bootcamp next week too14:49
i've played with karaf, but i don't understand it14:50
ok, lost interest in deploying to tomcat, ran into an NPE: https://gist.github.com/469363314:51
<ajs6f>That's werid. It can't build the repo because of something going wrong with the transaction manager.14:58
You want to make a ticket for that, or shoud I press on towards OSGi (which would change the whole build architecture anyway)?14:59
<cbeer>ajs6f: sorry, we had a door-alarm-interpreted-as-a-fire-alarm15:09
i'd do OSGi, i think
<ajs6f>No prob.
That makes sense to me, too. Get us on down the road down which we're already going.15:10
* JasonDGI leaves15:25
<ajs6f>Until I hear back from cbeer or the DGI contingent, I'm not going to do much more with the Atom-publication code. Maybe just some cleanup and adding tests. The real test is whether it can power GSearch in an Islandora install.15:26
<cbeer>sorry, let me take a whack at that. i got gsearch installed at least.15:27
<ajs6f>No hurry. I'm about to knock off for the day antyway.15:28
<cbeer>ok. then i may not get to it until next week15:29
i'm already confused about which config file to change
<ajs6f>To do what?15:35
<cbeer>ok, this is just stupid:15:39
[copy] Copying 36 files to /Users/gertschmeltzpedersen/f36/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal
[copy] Copying 1 file to /Users/gertschmeltzpedersen/f36/tomcat/webapps/fedoragsearch/WEB-INF/classes
<ajs6f>Oh, the GSearch config...15:41
yeah, that always weirded me out a bit too.
<cbeer>and i don't want to know why it needs to know where the objectStore is
<ajs6f>Lot of Ant scripting going on there, and config getting built into the classpath.
It doesn't!
<cbeer> #objectStoreBase must be the location of the objects of this repository.
objectStoreBase=${local.FEDORA_HOME}/data/objectStore
<ajs6f>That's a thing thats' been going around foever.
It doesn't really.
Put /tmp or soething in there.
It should run.
That goes back to when GSearch was built as a decorator on the Management module in Feora.15:42
Before the advnet of JMS eventing in Fedora.
<cbeer>i also don't see where to tell it how to find solr15:44
<ajs6f>Erm. I forget. I think it's in some kind of FgsConfigSolr type directory?15:45
I know you're looking for a file called indexer.properties.
<cbeer>yeah, i think jonathangee or JasonDGI get to take a swing at that
<ajs6f>Right on.15:46
They ought to find it easy
<cbeer>oh, here we go. in a different properties:
# FgsIndex: indexBase is the server base url, in case of Solr or Zebra.
hm. and there's a schema provided, but no solrconfig?15:47
<ajs6f>Do you want Gert to do _all_ of your work?! {grin}15:49
<cbeer>sensible defaults is all I ask
<ajs6f>I have a feeling that they're there, but just hard to find.15:51
But I really think you leave it up to the DGI dudes.
They'l get it done faster, because they do it all the time.15:52
<cbeer>swing and a miss. again.
guess i accomplished something for the day though
<ajs6f>I'm heading out myself. Talk y'all soon.15:56
* ajs6f leaves
* jonathangee leaves16:08
* avessey leaves16:10
* jonathangee joins16:12
<cbeer>eddies: ok, i've made a more itemized shopping list on https://wiki.duraspace.org/pages/viewpage.action?pageId=3464024516:24
should get us through long enough to figure out what we actually want to eat16:27
* jonathangee leaves16:29
* barmintor leaves17:38
* fasseg leaves19:45