You are here: Home > Blog > Authors > rich

Richard Watson

Mar 10, 2011

Zimbra Virus Quarantine

by Richard Watson — last modified Mar 10, 2011 11:08 AM

How to push a message through that triggered a false-positive on Zimbra's Virus detection

Our Zimbra email service is a great alternative to Microsoft Exchange, one of its built-in features being spam/virus protection.

One thing it currently lacks though is a full "quarantine" system, which is rarely a problem, but very occasionally a false positive is triggered by an attempt at sending a file securely falling foul of the "block encrypted archives" option. At this time it's handy to be able to send the mail on its way.

I had to do this today, and here's what I did:

  • Find the right mail in /opt/zimbra/data/amavisd/quarantine it will be called something like virus-1BghBzSYKd2E - this can involve finding the time of the original message in the logs and comparing with the timestamp on the virus file. In the unix world "ls -lhrt" is your friend here for listing the virus files in order of arrival. The location could be different on your install.
  • Deselect the option to block enrypted archives in Global Settings -> AS/AV (or remove filtering of the particular file type also in the Global Settings as appropriate).
  • On the server as user zimbra do "zmamavisdctl restart"
  • To send the mail on its way again: sendmail -t -i < virus-1BghBzSYKd2E
  • Look in /var/log/mail.log for the mail being sent ok and not still being rejected.
  • Reselect the block encrypted archives option so that it operates in future
  • On the server as user zimbra do "zmamavisdctl restart" to finally put things back to normal.

Email is a great tool but encrypted zip files are quite weak from a security perspective, so probably a better means of communication for a private file like this is to send by a secure means such as Dropbox or, if sending from Zimbra, to use the Zimbra Briefcase, or to use a more robust form of encryption for the email such as PGP.

 

Feb 17, 2011

Migration to Plone 4

by Richard Watson — last modified Feb 17, 2011 10:10 AM

How to upgrade to Plone 4

Cataract operations and gallbladder extractions might be the most common kinds of surgery on the NHS but our customers prefer a Plone migration. 

Plone.org has an online manual for those who like getting their hands dirty, but it's easy to see why many organisations like specialist help with this task.

Since the arrival of Plone 4 we've been fairly inundated with requests for information on how to upgrade sites (some as old as Plone 1.0.5) to it. Sometimes the older sites are actually simplest and quickest especially if they are sites that were setup for a particular purpose with only a few extra products.

It's fair to say that the addition of a 3rd party product which isn't supported in recent Plones can be the biggest stumbling block to a full migration. Another issue is a lot of bespoke templates, especially if the site's main template has been customised. This means that a lot of functionality may have to be recreated in a Plone 4 kind of way using new templates. These solutions aren't always expensive although some customers may take the opportunity to rethink the overall design and kill two birds with one stone.

Rarely it does happen that it's not feasible to migrate a whole site, and instead it's best to make a new Plone 4 site and try and move over as much content as possible. At this point Dexterity in Plone 4 is a real bonus - it means we can quickly create new content types to match the old ones and then write a script to convert the data from one to the other.

Sometimes there is some particular content that isn't worth the time taken to reproduce and a manual cut/paste is the best solution. However, to date we have succesfully moved at least 95% of any site content when requested and our customers are enjoying all the benefits of Plone 4.

For more details about migrating to Plone 4 please contact us.

 

 

Jan 31, 2011

Linux corrupt text console on Supermicro KVM Daughterboard

by Richard Watson — last modified Jan 31, 2011 10:35 AM

What to do if your text mode console doesn't show the correct graphics mode and the text is unreadable.

Back in the good old days, graphics were graphics and text was text. These days even something that looks like a text mode might be using the framebuffer and technically have a graphics resolution.

We encountered this problem recently when recommissioning a server.  To enable remote access should the server encounter a problem many of our servers have a KVM over IP daughterboard which allows console access if all else fails. This particular device also allows a remote user to hook up a local CD ROM or ISO image to allow a remote install to take place.

This was what we attempted recently with Ubuntu 10.4 (LTS). Unfortunately the graphics controller on the KVM daughterboard (in this case a Supermicro M3296) operated fine until the installer booted but then showed a corrupt display claiming to be 648x481 pixels in size. 

So, instead, on one of our datacentre visits, we reinstalled it in the conventional way, however afterwards the screen was still corrupt when viewing using KVM over IP.

The reason for corruption is because of a particular mode used by the linux console. This can be edited in the file

/etc/modprobe.d/blacklist-framebuffer.conf

 

and inserting

 

blacklist vga16fb

solves the problem.

I also believe that the kernel options 

vga=normal nomodeset 

should have the same effect during install, but we haven't tested that theory yet.

 

Nov 19, 2010

IOError: decoder jpeg not available

by Richard Watson — last modified Nov 19, 2010 11:55 AM

Moving a Zope instance made Plone sites lose image thumbnail generation. Seemingly odd PIL behaviour.

As often happens in hosting we recently moved some sites from one server to another. In this case a server had gone bang and so we rebuilt everything on a new server running the latest Ubuntu LTS.

Of course, one issue in moving the hosting was that the new server installation didn't come with the same versions of software installed as the old one did. In this case our Zope instance needs Python 2.3 - a fine vintage - but our new Ubuntu installation has Python 2.5 as the lowest available version for install by the package manager.

Thankfully, in the open source world it's easy to build older versions to run pretty much anywhere you want; so in a very short time we had the Zope instance up and running on our homemade Python 2.3 complete with Python Imaging Library (PIL). 

Or so we thought.

After a short while we noticed an issue that thumbnails were not being generated when new images were uploaded. In the log was found: 

File ".../lib/python2.3/site-packages/PIL/ImageFile.py", line 189, in load
d = Image._getdecoder(self.mode, d, a, self.decoderconfig)
File "...lib/python2.3/site-packages/PIL/Image.py", line 385, in _getdecoder
raise IOError("decoder %s not available" % decoder_name)
IOError: decoder jpeg not available

This error suggests that PIL had not been built with libjpeg support, however when building PIL it had output the following:

 

*** PIL CORE support not installed
*** TKINTER support not installed
--- JPEG support ok
--- ZLIB (PNG/ZIP) support ok
*** FREETYPE2 support not installed
*** LITTLECMS support not installed

which indicates that libjpeg and zlib were both found during the build. 

After a bit of rebuilding and head scratching the Plone community came to the rescue here:

http://plone.org/documentation/error/ioerror-decoder-jpeg-not-available

In our case we had simply omitted to specify the root of the libjpeg install (for us this is simply /usr because it was installed by the package manager).

So the crucial thing for us was to replace in PIL's setup.py :

 

JPEG_ROOT = None
ZLIB_ROOT = None
with
JPEG_ROOT = libinclude("/usr/")
ZLIB_ROOT = libinclude("/usr/")
 

It's a bit odd that it goes on and builds anyway with this missing but now we know about it we'll be looking out for it next time.

 

Sep 07, 2010

Google Wave is dead, long live Wave

by Richard Watson — last modified Sep 07, 2010 09:35 AM

Is the end of Google Wave the end of a new kind of communication, or just the end of the beginning?

A little over a year ago I blogged on the new phenomenon that was Google Wave. During that year we saw it gain popularity and all manner of extensions, plugins and desktop applications like Waveboard. As a company we used (and still use) it to write collaborative documents. Sadly, though, Wave would never reach its potential before Google decided to pull the plug on it.

On the face of it Wave was an obvious development - we all send messages around the world by various means, email, IM, forum, wiki, blog - so what we needed was a way to tie it all together - wasn't it? Well maybe, but the crucial thing that needed to happen didn't - at least not in time. Wave never saw its tentacles extend fully into the systems that we all know and already use, meaning that you had to convert yourself over to Wave in order to use it.

Another drawback was that there was no way yet (although it was promised for the future) to really use your own Wave server and have it hook into a larger Wave "cloud". That and the overall complexity of the Google interface seems to have resulted in Wave's premature death.

The odd thing now is that out of the ashes of Wave comes, yes, Wave. Google recently announced "Wave-in-a-box" which is ostensibly giving wave to the Open Source community so that in time everyone could host their own Wave server. This could be the best thing that happened to it. The Open Source community already manages development of the world's most popular web server - Apache, the most often recommended browser - Firefox, and provides free alternatives to commercial software in just about every sector.

The Google implementation of Wave was arguably too different and radical, too ahead of its time, but if the Open Source community can build on this basis in a modular way then, in time, Wave could still become the foundation of a new way of sending messages, an all-encompassing, open, distributed system that represents as much of a step forward as email has been over the last 20 or more years.

 

 

 

 

Feb 24, 2010

Recovering a Zope or Plone site that you can't access

by Richard Watson — last modified Feb 24, 2010 10:05 PM

Editing the ZODB via a python shell

Recently we had a case where someone changed the encoding on their Plone site using the portal_properties tool. This resulted in the site being unaccessible, even via the ZMI.


So, for posterity, here's how to recover from a fatal property setting:

1) Stop the instance (bin/instance stop or whatever)

2) Attach a python shell to the instance. There are a number of ways but in recent Zopes that ship with Plone (in this case Plone 3.3.1) the easiest is this:

bin/instance debug

This should run up a few lines and then settle down to a >>> prompt where you can type some python. You're now in a python shell and the ZODB is directly accessible via the "app" object.

3) Start a transaction. The right syntax had me foxed for a while, as it seems to have changed along the way, but the way to do it in recent zopes is like this:

>>> import transaction

It's important to do this before you start fiddling.

4) Make any changes you want. There are a few pages on things you might like to do to delete rogue objects etc, such as these:

http://www.zopelabs.com/cookbook/1054240694
http://wiki.zope.org/zope2/DebuggingWithIPythonAndOtherTips

However these are a bit old and don't help so much in this case as we need to change a property. So we do this:
 

>>> obj= app.unrestrictedTraverse("/Plone/portal_properties")

We can then look at a property like so:

>>> print obj.site_properties.default_charset
ASCII

Aha, so this is wrong. Let's change it:
 

>>> obj.site_properties.default_charset="utf-8"

And see if it's changed:

>>> print obj.site_properties.default_charset
utf-8

5) Very important step - if all went well don't quit the shell at this point, we need to commit like so:

>>> transaction.commit()

[If all didn't go well, then best to quit and not commit. That should leave things as they were before you started.]

6) You can then detach from the python shell with ctrl-d and see if your Plone site now works again.

Jul 28, 2009

Google Chrome and Open Source

by Richard Watson — last modified Jul 28, 2009 03:35 PM

Most people don't even know what an Operating System (OS) is - will they care about Chrome, even if it is Open Source?

Often I'm asked by friends with little knowledge of the world of computers to recommend a laptop or desktop machine for their home use. Although I'm still a fan of Linux, these days I nearly always suggest that they pick the Mac they most like the look of. A long string of questions and answers then ensues. Generally the enquirer's husband/mother/friend/uncle will have suggested some brand of PC running the latest incarnation of Windows, so my suggestion can come as a bit of a shock. Why would someone that appears to know what they're talking about even think about suggesting a computer that doesn't run the the programs that Auntie Betty's machine can?

It's into this world that Google Chrome has burst on the scene. Its tantalising open source goodness means that all us techies are itching to try it out, but does Auntie Betty want to use it? Will she even notice it?

In recent times the "netbook" has become a bit of a breakaway from the general home computing scene - often these things (generally the cheaper ones) are powered by alternative OSes (usually a shade of Linux). Strangely non-windows netbooks do sell, sometimes to ordinary people. When I went into the shop to buy one for my daughter I was summarily informed that I didn't want one of these because it didn't run Windows. When I retorted that this was one of its major selling points, I got a strange look as if to say that I know nothing about these things.

Maybe I do know nothing about what the average person wants. Maybe the ideal world would be one where everyone runs Microsoft and we all share the same viruses, sorry, I mean "binary compatible code". Probably when the ChromeOS netbooks appear the salesman I enountered will still try to push buyers onto a more expensive machine with "better compatibility".

Sadly, for all us geeks out there, this is where marketing can do what years of volunteer effort has failed to do - package up the advantages of open source for the uninformed user and gain market share in the face of a competitor who starts with a huge advantage.

After the launch of Android, Google should have a fair idea whether or not Auntie Betty will be getting a ChromeOS machine next and whether she will like it. I hope for her sake, and everyone else's, that they get it right.

 

 

 

 

Jun 02, 2009

Google - give us a wave!

by Richard Watson — last modified Jun 02, 2009 09:40 AM

Can Google's new masterpiece really take the world by storm?

It's a rare moment at Openia when we stop what we're doing to watch the same video on Youtube, and an 80 minute long one at that. Google's Wave presentation has such an air of occasion about it, that it would seem churlish not to take the time to watch. But is there any substance behind the understated-but-we-know-we're-good Google glitz? There are some pretty powerful applications out there, including many excellent web-based ones, so what exactly is it that makes Google Wave the "killer app" that so many claim it is?

First, it's important to recognise what Wave isn't - it's not one single new idea, it's a bunch of very small, but important, changes to a lot of other ideas which have been brought together. So the first thing we realise is that Wave owes a lot to other technologies. In so doing it stands on the shoulders of giants such as email, blogs, forums, social networking and instant messaging. (Even some of the "new" ideas, such as the ability to watch your correspondent as they are typing the individual letters and words of their message are not that original - "Phone" on VAX/VMS had such a feature back in the 1980s.)

Secondly, although it's a great piece of work, expertly delivered (from what we can tell from the video) it is in many ways completely inevitable. It was only a matter of time before blogs and email and messaging were integrated in some way - it is a technology waiting to happen. That it has happened in one big jump makes it more exciting, but you can rest assured that we would have got there eventually. That said it is reassuring to have Google at the helm at moments like this, rather than some of the alternatives.

But the part which makes this the app which will replace email, IM and many other messaging tools in all our lives is the fact that it is released as a protocol, and an open one at that. So anyone can write their own Google Wave compatible applications and talk to anyone else's. This is what makes Email the current primary choice for messaging - there is no need for everyone to use the same provider. Provider choice is key for planning communications, and Google are big enough to know that they can't be the only provider in the world without poisoning the well for all of us. One has to wonder whether if this technology had been constructed by Microsoft or Apple (for example) that it would have been used as a tool to try to secure market share, rather than making the world a better place for us all.

As a microblog, Wave might only be an incremental change from Twitter, and as a Forum/Social Network it's only a few thought processes removed from Multiply or Facebook, and as an Instant Messenger it might be practically the same as Jabber, MSN or Yahoo; it's the combination of these facets and, more importantly, the openness and ubiquity of them that makes Wave unique, as will be evident when the makers of Twitter, Facebook and others present their offerings in a Wave-like way.

And if this all works out as it should, Wave will become the "glue" of the web.