Tuesday, March 29, 2011

Cloning 10.04 server with clonezilla fails unless...

There seems to be an issue with cloning my Ubuntu 10.04 server and clonezilla.  When I tried to create a backup image of the disk I get the: "Something went wrong" message.  On line I found that the problem is probably with the i-nodes and the easiest way to solve this is to use clonezilla with the fsck option.
  1. Start like one normally does when cloning, but don't choose 'beginner mode' instead choose 'expert mode'.  Then you'll go though some similar screens but then you'll get to the advanced screens.  Always choose whatever option it defaults to, but one of the screens will have lots of options, one of which is "-fsck-src-part", select that and continue on and... things invariably work.
I wish I had recorded this on my blog the last time I had to solve it, but got it now!

Saturday, March 26, 2011

Developing alkisg's idea of synching table-servers, and no network cables...

I just got off of #ltsp with alkisg.  I needed further details on the hoped-for solution for next year set-up which wouldn't require using network cables but would require using at least 1 fast wireless access point and...  The following is a slight bit abridged (to remove extraneous content).

[3:44PM] dgroos: I'm wondering if you've had further thoughts about your idea on clustering over wifi.
[3:47PM] dgroos: Having 8 recycled P4's as elements of a cluster, connected via wifi, then each of these elements would be connected via ethernet to a few computers at the table.
[3:50PM] dgroos: Thus, the fat clients would be able to boot via cable as they are designed to do, but they wouldn't need network cables leaving the tables.
[4:09PM] alkisg:  Took a bit more :) reading...
[4:10PM] alkisg: dgroos: sounds reasonable
[4:10PM] alkisg: And you would have 1 authentication server and a common nfs home for everyone?
[4:10PM] alkisg: Or local nfs homes on those clusters?
[4:12PM] dgroos:  I wasn't sure if one would want 1 powerful server as the backbone so to say of all of the low-power servers in the cluster, and...
[4:12PM] dgroos: if there would be memory limits since the p4 elements are 32 b arch.
[4:13PM] alkisg:  The fat client servers don't need CPU. A bit of RAM, to cache parts of the fat nbd image, and a fast network
[4:13PM] dgroos:  or, if there would be compatability issues with a 64b machine along with several 32 b machines.
[4:13PM] alkisg:  No, it doesn't matter at all, the fat client servers are only serving an nbd image etc
[4:14PM] alkisg: They could even be... mac or arm
[4:14PM] dgroos:  Wow--so I could get some very small hardware to do the job--fits in the table better.
[4:16PM] dgroos: Would I even need a strong server in the cluster?  I'm thinking of storing the home folders there--is that how you would recommend it?
[4:16PM] alkisg:  Do the kids change places? If you can force them to use the same cluster each time, then yes, I'd recomment putting both authentication + homes there
[4:17PM] alkisg: And only use wifi for internet access
[4:17PM] alkisg: I.e. 1 cluster == 1 independed fat ltsp setup
[4:17PM] alkisg: Of course you can clone the servers for installation
[4:19PM] dgroos:  Kids would definitely change places several times throughout the year--practice in working with new teams etc.
[4:19PM] alkisg:  Then you'd need /home over wifi
[4:20PM] alkisg: Or some clever way of moving /home/username every time a student moves to a different cluster
[4:20PM] dgroos:  OK that doesn't sound like it would generally be an issue.  Would it get loaded locally on the fat client when a student logs in?
[4:20PM] alkisg:  The fat client would access /home/username with sshfs by default, or if you prefer with nfs
[4:21PM] alkisg: That's over wifi
[4:21PM] alkisg: So for 10 students trying to access /home over wifi the bandwidth would drop a lot
[4:21PM] dgroos:  'k
[4:21PM] alkisg:  How many students using a single wifi access point? And what speed? 50 mbps?
[4:21PM] alkisg: (seats, not students)
[4:22PM] dgroos:  I suppose we would use n networking standard, not sure where it's at...
[4:23PM] alkisg:  That would suffice for a lot of seats
[4:24PM] dgroos:  And, I wonder if I could put a couple of access points, at least 15 (for 2-1 ration) or better yet 30 separate users at any time in a classroom.
[4:25PM] alkisg:  I think I'd implement the other option, the "syncing /home/username locally when the student logs on"
[4:26PM] alkisg: locally == in the cluster server
[4:26PM] dgroos:  Would I need a regular (dual-core xeon) server if I had all home folders on it and was using pentium 4 fat clients?
[4:26PM] alkisg:  No, you wouldn't need a big server at all
[4:26PM] dgroos:  OK, I wonder how difficult some script like that would be?
[4:27PM] alkisg:  Not very much, writing such a script + debugging it should take less than a day
[4:27PM] dgroos:  would this work with sch-scripts, that fancy program coming of that famous app-shop? ;)
[4:27PM] alkisg:  (it would be best if it only synced when the student actually changed cluster, not every time)
[4:28PM] alkisg: Hehe
[4:28PM] dgroos:  sure.
[4:28PM] alkisg:  Each cluster could have its own versions of sch-scripts, you'd need a master one?
[4:29PM] dgroos:  When you say each cluster, you mean each element of a cluster?
[4:29PM] alkisg:  Would be possible too, but it would need some tweaking with the different servers
[4:29PM] alkisg: With cluster I mean a table consisting of 1 fat-client-server and 4-8 fat clients
[4:29PM] alkisg: Maybe not the right word, I can call it "table" from now on :)
[4:30PM] dgroos:  OK I was thinking that each of the 'servers' at the table would unite and be part of a single 'cluster'--I didn't have the under-the-hood visualized, however.
[4:31PM] alkisg:  Maybe you want to have a look at ltsp-cluster? Never looked at it, don't know if it'll suit you
[4:32PM] alkisg: I'd just use 1 master server for central authentication and for nfs homes, which would be rsynced locally when needed
[4:32PM] dgroos:  That's what I was imagining when you said cluster!
[4:33PM] alkisg:  I.e. the fat servers on the tables wouldn't have any user accounts at all
[4:33PM] dgroos:  And the fat client images would be at the table-servers...
[4:34PM] alkisg:  Right, just for speed
[4:34PM] alkisg: You wouldn't need to maintain those, you could maintain the fat image on the master server, and copy it to the table servers when updated
[4:36PM] dgroos:  Right, so as you say, the entire master server (master server? sounds paradoxical!) would have all the table servers rsync to it for everything BUT the home folders...
[4:37PM] alkisg:  No, not for everything. They wouldn't need /opt/ltsp/i386
[4:37PM] alkisg: They'd only need /opt/ltsp/images/i386.img
[4:37PM] alkisg: They wouldn't need ubuntu-desktop or even X
[4:37PM] alkisg: But they'd need some gigabytes for /home, for the students that connected there
[4:39PM] alkisg: So, ltsp-update-image on the server should also rsync /opt/ltsp/images/i386.img to the client,
[4:39PM] dgroos:  OK, right, each night or whenever, I could rsync the master home directories with the home directories on the table-server (for just the students who sit at that table).
[4:39PM] alkisg:  and a login script should rsync /home/username to the table server
[4:39PM] alkisg: *sorry I said client above, I meant table server
[4:41PM] dgroos:  This is almost sounding like a done deal (ignorance is bliss :D ).
[4:41PM] alkisg:  The syncing part is a bit more complicated than that
[4:42PM] alkisg: I.e. when logging off, the local copy is more recent than the "master" on
[4:42PM] alkisg:  So I'd prefer to use a login syncing script, which would sync from the last logon server
[4:43PM] dgroos:  Isn't there an option with rsync to take the most recent version?
[4:43PM] dgroos: "last logon server"?
[4:43PM] alkisg:  Kid sits on table A
[4:43PM] alkisg: The next hour he sits on table B
[4:44PM] alkisg: Describe to me how you got his documents transfered to table B :)
[4:45PM] dgroos:  At this point in the project that senario wouldn't happen (they've only got this in their science class) but I see your point.  So,
[4:47PM] dgroos: are you envisioning when a user logs out the changes get exported from the table server to the home folders on the master server, then when that student logs in the next day or at a different table they get updated back at the table server?
[4:48PM] alkisg:  (1) how often would kids change tables? and (2) do you keep daily backups for /home?
[4:50PM] dgroos:  1) every couple of weeks in my class but in come teachers classes it could be almost every day and in others 1 time a month.  2) I haven't kept daily backups but I could I guess.
[4:51PM] alkisg:  Based on your answers, I'd use the "last logon server" solution I described above,
[4:51PM] dgroos:  I didn't really get that solution...
[4:52PM] alkisg:  In that solution, there is no /home on the "master server". All /home/usernames are local to the tables
[4:52PM] alkisg: So the kid logs on to table A, and gets his /home/username folder there
[4:52PM] alkisg: The next day he logs on again to table A. No syncing happens at all. Very very fast.
[4:53PM] alkisg: The next day he logs on to table B.
[4:53PM] alkisg: The script sees that the last logon location was on table A.
[4:53PM] alkisg: So it rsyncs directly from table A to table B, and updates the entry about the most recent login, which is now on table B.
[4:53PM] NeonLicht:  And what happens if tablet A is off/broken/lost?
[4:54PM] alkisg:  Then a clean folder is used, or, if an old one is available, the old one. But the user should be prompted on that case.
[4:55PM] alkisg: The same would happen with the "master server" /home too, it's not something specific to this solution
[4:55PM] alkisg: And it's even better with /home over wifi/nfs, because now there's at least the option for the kids to work in case the "master /home" would be down
[4:56PM] alkisg: (and faster too)
[4:56PM] alkisg: *better than, not better with
[4:56PM] dgroos:  So, there would be a complete set of home folders at each table-server, however, the only ones up-to-date would be the home folders of the students who last logged in at that table,
[4:56PM] alkisg:  Exactly
[4:56PM] NeonLicht:  The master can use a NAS, with RAID, or ZFS. 
[4:57PM] NeonLicht: A sync back to the server after use could be a good idea, I think.
[4:57PM] dgroos:  and when a student moved from one table to the next there would be updating of the student's new table-server home folder.  Slick!
[4:58PM] alkisg:  dgroos: are you planning on frequently moving the tables, i.e. is the possibility of the "last logon table missing" high?
dgroos:  NeonLicht: The NAS would be for backup purposes then? 
[5:02PM] alkisg:  Being a teacher that also doesn't backup every day... It would be enough for me to have a backup from last week in table "A", even in the case where the hard disk in the most recent logon  table "B" was destroyed