Archives for May 2009

My AFP problem

2009/05/18 By staze

Since January of this year, I’ve been actively seeing AppleFileServer crash regularly on a server at work. This server is our primary student account server, which at any given time has about 40-80 students logged in (network home directories).

Many days, AFP crashes several times. Every time, it’s the same error: kern_protection_failure. The thread that crashes is always talking about ByteRangeLockTreeKey. The only good thing about this problem, is seemingly AFP comes back up, and people’s computers reconnect (go autofs!). But this is a very poor consolation prize since for some people, this does cause a problem (anyone with Mail open usually gets an error about not being able to access their inbox, and do they want to rebuild, or quit, and some others occasionally get Final Cut project file corruption (this is rare, and only seems to impact those that have their autosave vault set to their home directory, and not the local HD)).

So, Apple was notified about this, officially, on Jan 22nd, 2009. Ticket number 6517425. After getting back to me and asking for some follow up info, they proceeded to roll the ticket into another one (6237420). This ticket, apparently, was not related, and after telling our Sales Engineer about this, he had them un-merge the tickets. Apple then rolled my bug into another ticket, 5859645. An even older ticket! From what I’ve gathered, this ticket may be related to some lower level issue than AFP… either filesystem level (perhaps ACLs?!?, or even general I/O level).

All the while, I am in contact with someone in Minnesota who is having my same issue, and has also opened tickets (and has the luxury of having AppleCare for 10.5 server (the high end AppleCare to boot). He had two open case numbers with them. He even had a regional service engineer come by and take a look at this system, which he said was set up correctly, and there’s nothing more they could do to help alleviate the problem until a patch was available.

So, also during this time, someone from London contacts me and says he’s having the same issue as well, and has a Developer account (pay for), so he tries a beta of 10.5.7. It does not fix the issue. Around this time, I downgrade to 10.5.4 hoping the issue will be lessened (long story short, it isn’t). But, a few weeks later, the gent from London says he’s fixed his problem by removing the “deny all” acl from all his share points and folders within share points. The “deny all” acl was added around 10.5.4 or so to mitigate something… no one’s sure what. Anyway, he then tells Apple about this “fix” and they reply that it’s an “unacceptable workaround” and that they’re working on a fix. This was April 9th he did this.

Well, so, 10.5.7 dropped last Tuesday (May 12th, 2009). I installed it on the server experiencing the issue Friday night, at about 2am. I didn’t have a single crash until Sunday, May 17th, 2009, at 5:52pm. Same exact error.

So, not only was Apple notified AT LEAST 110 days prior to 10.5.7 shipping, but they were notified of an actual “fix” about 33 days before hand. I really wish Apple’s bug database was public, so that I could post links to my bugs, but, alas it is not.

However, here are a few threads on the issue:

http://www.afp548.com/forum/viewtopic.php?showtopic=23311

At this point, I’m going to start actively poking buttons and prodding people until I get an answer. The last email I sent to [email protected] resulted in the “pat”, “There is no new information at this time”. What a load of horse crap. They know of at least one “option”… the least they could do would be to educate someone having this issue about that “fix” and it’s repercussions. Given the amount of time that 10.5.7 took to hit the street, and how far in advance I notified them about this bug, I have very little hope this will get fixed before 10.6. If we’re lucky, we’ll see the fix back ported, but I doubt it.

To cap this all off, the main reason I’m posting this is for posterity, as well as the hope that anyone else that has this bug can actually see they’re not alone! And that they can contact Apple and say “hey, I have some bug numbers here of others having this issue”. If you are having this issue, please, don’t hesitate to contact me and I’ll work to get you in contact with others having this issue, or with someone at Apple that will actually listen.

UPDATE 1: Today I got a call from the local Education SE, who has created an escalation of this issue. Assuming it gets signed off by his boss, I should be hearing from Apple Engineering in the next few days… which is good since AFP crashed 5 times today. I have decided, in the interim, to remove the “group:everyone deny delete” ACL from many of the home folders on the server. Hopefully this will ease the problem. We’ll have to see. And I’ll post more once I hear from Engineering.

Quick update

2009/05/15 By staze

Just thought I’d post and give a quick update on the last two days.

Yesterday, Tara and I tackled a long overdue project that I’ve been dreading since it involved a bunch of time in the crawl space (well, 30-45 minutes). We replaced the dryer exhaust vent. The old one was one of those wedge shaped ones that directed all the flow downward, and had a flapper below the “wedge” that keeps outside air and critters out. Well, the vent was situated right above the ground (it comes out from one of the crawl space vents). So, the vented air goes right into the ground, and if there was any soil/debris buildup in the herb garden, it would block the flow of exhaust. The other problem being the flapper had long since stopped working/been clogged with lint and debris. So outside air came down the vent into the laundry room, making it even colder. So, we replaced the vent exhaust with a modern louver style (one of these). The advantage is supposedly they have better flow, they don’t blow down into the ground, and they actually close, so no more air coming down the vent pipe. YAY!!!!

Then today, the other project I’ve been meaning to do: repair the front sillcock (a frost-free Nibco 90). Back when we bought the house, I noticed that if the sillcock is open, but there is no flow (a sprayer on the hose, a Y with both sides closed, etc), it would leak at the stem. Opening it up, the packing was all torn up. So, I tried to rebuild it at the local hardware shop with various parts. Afterward, it wouldn’t leak, but it was a bitch to turn on and off, and it didn’t turn on all full flow unless you opened the valve all the way. So, this year, while wandering around the local hardware store, I noticed they had rebuild kits for “Fu San” Frost free sillcocks (not that I can find such a thing exists… the receipt, however, says “Flor Repair Kit”) that looked like they had the right parts. So, I pulled it apart again, and brought in the stem. Then at the store, put it all together with the new parts. Similar to this. Turned the water back on, and viola, it works like brand new! And the vacuum breaker seems to work again (it was kinda weird between the first time I fixed it, and now). Cost of repair kit, $2.99.

Other than that, Tara and I are going to be on cleaning duty the next day or so since Tara’s mom and aunt’s are coming to visit. Hopefully we’ll be ready for them. We’ve been struggling to control a sugar ant problem in our guest room (of all places, that doesn’t ever have food in it). Here’s hoping it’s somewhat better by then.

So, there it is. Have a great weekend!

More yard/house care

2009/05/13 By staze

So, last week before leaving for Portland, I fertilized the blueberries again. They’re looking quite nice, though I’m still confused why only two are flowering. My hope is that with some judicious pruning this winter, we can get them all flowering (from reading, flowers are borne on 2 year old wood, so it could be that there’s just too much young wood on them. *shrugs*). Then yesterday, we went and picked up a yard of doug-fir sawdust. Turns out, this was way too much. So, we now have sawdust about 3-4″ thick under the berries, as well as all the way to the fence. =P Oh well, it should really keep the weeds down, and will certainly make the blueberries happy.

Now I’m reading up on lawn care. The front yard at our place looks the best it has, well, ever. It’s extremely healthy, very few weeds, and all and all very nice (bit of crab grass, but I hope to take care of that soon). The back, on the other hand, not as nice. TON of weeds on the north side, I’m guessing due to the lower light levels. So, I’ve been trying to weed-n-feed there. Hopefully that too will work.

We also finally got around to removing the very sick/dying rhododendron that had been planted next to the front door (in a built-in planter). It doesn’t get any rain in that location, and because of the make-up of the planter (mortar, bricks, etc), it was probably WAY too alkali for it anyway. I put it in a 25gal nursery container with a rough 70/30 mix of peat/perlite, some acid loving osmicote, and a good drench of Superthrive. Hopefully it’ll perk up. The one that was sick and dying elsewhere on the lot I did the same thing to last year, and it’s extremely healthy now.

Tonight I’m going to double the RAM in the web server/file server/energy monitor, etc. Hopefully everything will work. I tried turning on mod_deflate on the system yesterday, and was dismayed by the performance. I hope to just enable it on some of the stuff I’m working on that will be a lot of data (further using google visualizations for power use, which 3 days of data would amount to 700KB uncompressed).

Mac OS 10.5.7 came out yesterday while I was getting sawdust, and I’ve installed it at work. So far, so good. They said they fixed some error with Dvorak keyboards, but I never noticed anything. My hope against hope is that it will fix the longterm issue we’ve had with AFP crashing on the server. Somehow, I doubt it.

This month will be the first time I can check my utility bill against the TED. I’m hoping it’ll be close… I really don’t want to have to call the local utility company to check my meter, though a newer digital one would be sweet. PGE up in Portland has been going around replacing analog ones with digital ones. I’m somewhat jealous.

At work we’re writing up a proposal for what amounts to “grant” money to offset probably budget cuts this coming year. I’m really really hoping to get 1-2 servers out of the deal. 2 servers would let me switch all critical systems over to Intel Xserves, and jettison 5 original Xserve’s. 1 server would let me upgrade the mail server and implement CalDAV. I would still need to dual purpose a machine to take up the slack from not getting the 2nd server… so, I’m hoping for 2. Mind you, this should also lower our power usage.

That’s all for now, since I can’t think of anything else.

Links for future reference:
http://ir.library.oregonstate.edu/xmlui/bitstream/handle/1957/19232/ec1521.pdf
http://extension.oregonstate.edu/catalog/details.php?sortnum=0616&name=Lawns

« Previous Page