Saturday, March 26, 2011

Developing alkisg's idea of synching table-servers, and no network cables...

I just got off of #ltsp with alkisg.  I needed further details on the hoped-for solution for next year set-up which wouldn't require using network cables but would require using at least 1 fast wireless access point and...  The following is a slight bit abridged (to remove extraneous content).

[3:44PM] dgroos: I'm wondering if you've had further thoughts about your idea on clustering over wifi.
[3:47PM] dgroos: Having 8 recycled P4's as elements of a cluster, connected via wifi, then each of these elements would be connected via ethernet to a few computers at the table.
[3:50PM] dgroos: Thus, the fat clients would be able to boot via cable as they are designed to do, but they wouldn't need network cables leaving the tables.
[4:09PM] alkisg:  Took a bit more :) reading...
[4:10PM] alkisg: dgroos: sounds reasonable
[4:10PM] alkisg: And you would have 1 authentication server and a common nfs home for everyone?
[4:10PM] alkisg: Or local nfs homes on those clusters?
[4:12PM] dgroos:  I wasn't sure if one would want 1 powerful server as the backbone so to say of all of the low-power servers in the cluster, and...
[4:12PM] dgroos: if there would be memory limits since the p4 elements are 32 b arch.
[4:13PM] alkisg:  The fat client servers don't need CPU. A bit of RAM, to cache parts of the fat nbd image, and a fast network
[4:13PM] dgroos:  or, if there would be compatability issues with a 64b machine along with several 32 b machines.
[4:13PM] alkisg:  No, it doesn't matter at all, the fat client servers are only serving an nbd image etc
[4:14PM] alkisg: They could even be... mac or arm
[4:14PM] dgroos:  Wow--so I could get some very small hardware to do the job--fits in the table better.
[4:16PM] dgroos: Would I even need a strong server in the cluster?  I'm thinking of storing the home folders there--is that how you would recommend it?
[4:16PM] alkisg:  Do the kids change places? If you can force them to use the same cluster each time, then yes, I'd recomment putting both authentication + homes there
[4:17PM] alkisg: And only use wifi for internet access
[4:17PM] alkisg: I.e. 1 cluster == 1 independed fat ltsp setup
[4:17PM] alkisg: Of course you can clone the servers for installation
[4:19PM] dgroos:  Kids would definitely change places several times throughout the year--practice in working with new teams etc.
[4:19PM] alkisg:  Then you'd need /home over wifi
[4:20PM] alkisg: Or some clever way of moving /home/username every time a student moves to a different cluster
[4:20PM] dgroos:  OK that doesn't sound like it would generally be an issue.  Would it get loaded locally on the fat client when a student logs in?
[4:20PM] alkisg:  The fat client would access /home/username with sshfs by default, or if you prefer with nfs
[4:21PM] alkisg: That's over wifi
[4:21PM] alkisg: So for 10 students trying to access /home over wifi the bandwidth would drop a lot
[4:21PM] dgroos:  'k
[4:21PM] alkisg:  How many students using a single wifi access point? And what speed? 50 mbps?
[4:21PM] alkisg: (seats, not students)
[4:22PM] dgroos:  I suppose we would use n networking standard, not sure where it's at...
[4:23PM] alkisg:  That would suffice for a lot of seats
[4:24PM] dgroos:  And, I wonder if I could put a couple of access points, at least 15 (for 2-1 ration) or better yet 30 separate users at any time in a classroom.
[4:25PM] alkisg:  I think I'd implement the other option, the "syncing /home/username locally when the student logs on"
[4:26PM] alkisg: locally == in the cluster server
[4:26PM] dgroos:  Would I need a regular (dual-core xeon) server if I had all home folders on it and was using pentium 4 fat clients?
[4:26PM] alkisg:  No, you wouldn't need a big server at all
[4:26PM] dgroos:  OK, I wonder how difficult some script like that would be?
[4:27PM] alkisg:  Not very much, writing such a script + debugging it should take less than a day
[4:27PM] dgroos:  would this work with sch-scripts, that fancy program coming of that famous app-shop? ;)
[4:27PM] alkisg:  (it would be best if it only synced when the student actually changed cluster, not every time)
[4:28PM] alkisg: Hehe
[4:28PM] dgroos:  sure.
[4:28PM] alkisg:  Each cluster could have its own versions of sch-scripts, you'd need a master one?
[4:29PM] dgroos:  When you say each cluster, you mean each element of a cluster?
[4:29PM] alkisg:  Would be possible too, but it would need some tweaking with the different servers
[4:29PM] alkisg: With cluster I mean a table consisting of 1 fat-client-server and 4-8 fat clients
[4:29PM] alkisg: Maybe not the right word, I can call it "table" from now on :)
[4:30PM] dgroos:  OK I was thinking that each of the 'servers' at the table would unite and be part of a single 'cluster'--I didn't have the under-the-hood visualized, however.
[4:31PM] alkisg:  Maybe you want to have a look at ltsp-cluster? Never looked at it, don't know if it'll suit you
[4:32PM] alkisg: I'd just use 1 master server for central authentication and for nfs homes, which would be rsynced locally when needed
[4:32PM] dgroos:  That's what I was imagining when you said cluster!
[4:33PM] alkisg:  I.e. the fat servers on the tables wouldn't have any user accounts at all
[4:33PM] dgroos:  And the fat client images would be at the table-servers...
[4:34PM] alkisg:  Right, just for speed
[4:34PM] alkisg: You wouldn't need to maintain those, you could maintain the fat image on the master server, and copy it to the table servers when updated
[4:36PM] dgroos:  Right, so as you say, the entire master server (master server? sounds paradoxical!) would have all the table servers rsync to it for everything BUT the home folders...
[4:37PM] alkisg:  No, not for everything. They wouldn't need /opt/ltsp/i386
[4:37PM] alkisg: They'd only need /opt/ltsp/images/i386.img
[4:37PM] alkisg: They wouldn't need ubuntu-desktop or even X
[4:37PM] alkisg: But they'd need some gigabytes for /home, for the students that connected there
[4:39PM] alkisg: So, ltsp-update-image on the server should also rsync /opt/ltsp/images/i386.img to the client,
[4:39PM] dgroos:  OK, right, each night or whenever, I could rsync the master home directories with the home directories on the table-server (for just the students who sit at that table).
[4:39PM] alkisg:  and a login script should rsync /home/username to the table server
[4:39PM] alkisg: *sorry I said client above, I meant table server
[4:41PM] dgroos:  This is almost sounding like a done deal (ignorance is bliss :D ).
[4:41PM] alkisg:  The syncing part is a bit more complicated than that
[4:42PM] alkisg: I.e. when logging off, the local copy is more recent than the "master" on
[4:42PM] alkisg:  So I'd prefer to use a login syncing script, which would sync from the last logon server
[4:43PM] dgroos:  Isn't there an option with rsync to take the most recent version?
[4:43PM] dgroos: "last logon server"?
[4:43PM] alkisg:  Kid sits on table A
[4:43PM] alkisg: The next hour he sits on table B
[4:44PM] alkisg: Describe to me how you got his documents transfered to table B :)
[4:45PM] dgroos:  At this point in the project that senario wouldn't happen (they've only got this in their science class) but I see your point.  So,
[4:47PM] dgroos: are you envisioning when a user logs out the changes get exported from the table server to the home folders on the master server, then when that student logs in the next day or at a different table they get updated back at the table server?
[4:48PM] alkisg:  (1) how often would kids change tables? and (2) do you keep daily backups for /home?
[4:50PM] dgroos:  1) every couple of weeks in my class but in come teachers classes it could be almost every day and in others 1 time a month.  2) I haven't kept daily backups but I could I guess.
[4:51PM] alkisg:  Based on your answers, I'd use the "last logon server" solution I described above,
[4:51PM] dgroos:  I didn't really get that solution...
[4:52PM] alkisg:  In that solution, there is no /home on the "master server". All /home/usernames are local to the tables
[4:52PM] alkisg: So the kid logs on to table A, and gets his /home/username folder there
[4:52PM] alkisg: The next day he logs on again to table A. No syncing happens at all. Very very fast.
[4:53PM] alkisg: The next day he logs on to table B.
[4:53PM] alkisg: The script sees that the last logon location was on table A.
[4:53PM] alkisg: So it rsyncs directly from table A to table B, and updates the entry about the most recent login, which is now on table B.
[4:53PM] NeonLicht:  And what happens if tablet A is off/broken/lost?
[4:54PM] alkisg:  Then a clean folder is used, or, if an old one is available, the old one. But the user should be prompted on that case.
[4:55PM] alkisg: The same would happen with the "master server" /home too, it's not something specific to this solution
[4:55PM] alkisg: And it's even better with /home over wifi/nfs, because now there's at least the option for the kids to work in case the "master /home" would be down
[4:56PM] alkisg: (and faster too)
[4:56PM] alkisg: *better than, not better with
[4:56PM] dgroos:  So, there would be a complete set of home folders at each table-server, however, the only ones up-to-date would be the home folders of the students who last logged in at that table,
[4:56PM] alkisg:  Exactly
[4:56PM] NeonLicht:  The master can use a NAS, with RAID, or ZFS. 
[4:57PM] NeonLicht: A sync back to the server after use could be a good idea, I think.
[4:57PM] dgroos:  and when a student moved from one table to the next there would be updating of the student's new table-server home folder.  Slick!
[4:58PM] alkisg:  dgroos: are you planning on frequently moving the tables, i.e. is the possibility of the "last logon table missing" high?
dgroos:  NeonLicht: The NAS would be for backup purposes then? 
[5:02PM] alkisg:  Being a teacher that also doesn't backup every day... It would be enough for me to have a backup from last week in table "A", even in the case where the hard disk in the most recent logon  table "B" was destroyed

No comments:

Post a Comment