Category Archives: Software Development

Posts about software development. Generally I use Java, PHP, and Python for development but occasionally I delve into other things as well.

SSHFS

If you’re a web or software developer, chances are you’ve been frustrated before by the inconvenience of having to SSH into the remote server all the time to work on files. You can’t use the cozy development environment you set up on your own machine. You have to use the server’s tools. (If you really like command line vi then this post isn’t for you).

What if the server doesn’t have the programs installed that you need to do your thing? If you don’t have sufficient access, you may have to get the admin to install the programs for you, and often times they won’t.

Wouldn’t it be nice if you could just mount the server drive and work on the site like they were sitting on your local hard drive? I just discovered SSHFS, which allows you to do just this.

Apparently SSHFS (secure shell file system) has been available for some time on unix/linux, but there was no equivalent on Mac OS X until recently: The MacFUSE project enables SSHFS and many other types of file systems to be mounted just like they were a local hard drive on Mac OS X. Its as simple as opening SSHFS, entering your connection information and your server hard drive will be mounted in finder like it is a local drive. You can run all of your favorite programs on the files.

In my case, what I needed to do was run a patch using then patch command. The server didn’t have patch installed, so I mounted the server using SSHFS and used my local version of patch to patch the files. What a time saver!

This ranks as the best thing since sliced bread in my books.

CAS4PAS Mission – The further adventures of Plone 2.5

Okay.. I’m still at this Plone 2.5 upgrade thing. The early reports saying that the upgrade from Plone 2.1 to 2.5 is a snap are indeed correct. Unless you are using CAS for authentication.

Since Plone 2.5 uses PAS (Pluggable Authentication Service) which is very different than the way 2.1 handled authentication, it is necessary to do away with the old way to do CAS auth.

Begin rant ….

The single biggest complaint about the Plone/Zope community is that they think that backward compatibility is optional. They don’t think twice about breaking old code in favour of new features. Undoubtedly PAS is a superior way of handling authentication and permissions than the old way, but couldn’t they make it backwards compatible with the old way of doing things. It is ridiculous to think that every time I have to upgrade to a new version that I have to spend a week or two feeling around the bugs and bases of the code and configuration just trying to make it work again. This paragraph is written out of anger and frustration…. bahh!!

End rant …

Okay, back to rational thinking. Here is what I have done so far:

  1. Downloaded CAS4PAS 1.0.0-1 and installed in the Products directory.
  2. Restarted Zope and added a CAS Helper to the acl_users folder of my Plone site – then copied the settings across.
  3. Downloaded and installed the new version of PloneCASLogin (because the old version doesn’t work with PAS).
  4. Went to a different browser and tried to log in using CAS.
  5. The login button takes me to the CAS login page OK, but when I return to the Plone site, it says that there was a sign-in failure.

At this point I went back to the download pages for the Plone CAS Login and CAS4PAS products to see if I was missing anything. I noticed a patch to make it compatible with CAS 2 (not sure if my server is CAS 1 or 2, but I thought I might as well download it anyway to see if that was the problem.

  1. Downloaded and ran the patch
  2. Restarted the server and tried to log in — same problem!
  3. Scoured the internet for information from others having the same problem. Found one helpful post here.
  4. It suggested that I try to disable the “Challenge” feature of the credentials_cookie_auth in acl_users.
  5. Same problem!

That is where I sit right now… No clues, so I’ll dig deeper into the code and see what I can find….

Plone 2.5 migration Chronicles

Okay.. the time has come for us to migrate from Plone 2.1 to Plone 2.5 on the Faculty web site. The word is that this migration should be much easier than the migration was from 2.0 to 2.1.. so here is a little log of my experience:

  • Given that requirements for Plone 2.5 include Zope 2.8.7+ or 2.9.4+ and we are running 2.8.5, I put in a request to the network support guys to upgrade Zope. They upgraded to 2.10.1 (which would seem to meet the requirement of 2.9.4+) and gave me the reigns.
  • Tried migrating the dataface site using the portal_migration tool. The migration appeared to be successful, so I proceeded to do a version migration on the portal_atct tool. That migration also appeared to be successful. Wow! Was that ever easy! But wait a minute …..
  • When I tried to visit some of the pages of the migrated dataface site, I received “404 resource not found” errors. Something went wrong, but what?
  • I then took a step back and tried something simpler: adding an empty plone site using the ZMI on a default Plone 2.5 install. The installation failed with errors.
  • I began to suspect that Plone 2.5 was not compatible with Zope 2.10, so I did some googling and found out that indeed it does not work. The requirement of 2.9.4+ actually means 2.9.x where x >= 4 – and NOT x where x > 2.9.4.
  • Then I got another bright idea: The requirements listed on the plone site for Plone 2.5 were identical to the requirements for Plone 2.1.4. We were running Plone 2.1.2 fine on our Zope 2.8.5 box so I thought maybe the 2.8.7+ requirement might be lax on the last digit, and that 2.8.5 would work. WRONG! It didn’t work.
  • Sent in a message to the network guys asking for a different Zope install (we’ll eventually have to upgrade to Zope 2.10 for Plone 3, but that won’t be for a while.
  • Now with my newly installed Zope 2.9.6 instance I was ready to go to town.
  • I ran a migration on the dataface site using the portal_migrations and portal_atct tools, and tested the site out. The migrations went smoothly (took about 20 minutes, but everything works tickity boo).
  • The Faculty site is another story. We are using CAS authentication on the site so that users can use the SFU single-sign on features and we don’t have to handle passwords. Apparently Plone 2.5 using PAS (Pluggable Authentication Service) which is quite different than the old system and doesn’t support migration of CAS User Folders. Hence the migration in portal_migrations failed.
  • Googling on the internet led me to a number of pages describing this issue. Apparently the current workaround for this is to delete the CAS User Folder and do the migration, then install CAS4PAS – an alternative CAS product that works with Plone 2.5. The problem is then I think we may lose all of the user roles, groups and permissions. 🙁
  • To be continued ….

dhtmlXGrid, open source but painfully restricted

I have been attempting to incorporate dhtmlxGrid into Dataface to spice up the editing of related data on web forms. Since Dataface is open source under GPL, any library that I choose to include must also be open source and compatible with GPL. dhtmlxGrid standard edition fits this criterion so I decided to take it for a spin.

Unfortunately the standard edition appears to be no more than a trial version of the software as it does not contain any ability to extract data from the grid (that I can see). Without this ability, it is impossible to actually save, submit, or send data from the grid to a database or a form. What good is a grid like this if it cannot be saved? All of the features that would enable saving data are reserved for the professional edition which has quite a restrictive commercial license which is, safe to say, not compatible with GPL.

I do however give them props for releasing the standard edition under GPL. They could have just released a trial version with no distribution rights at all. As it is, at least I can dig into the code and add the necessary functionality myself.

I am working with the Firefox Javascript debugger to try to see where the data is hidden. This will be a little time consuming, but the end result will be worth it, I think.

RSS Replacing Mailing Lists

I have recently discovered the joy of RSS subscriptions first hand.  Before RSS, the only way to stay informed of the news and events that interested me was to subscribe to the appropriate mailing list.  This resulted in a cluttered mail box.  Last week I downloaded an RSS news reader for OS X called Vienna, and I started subscribing to the RSS feeds of the web sites and message forums that I enjoy.  Wow, is this empowering.  No longer do I have to spend an hour surfing the web to see if there is anything new on my favourite sites – or on the Dataface forum.  Now I just have to check Vienna and see if there are any new posts.

DHTMLXGrid Part I

    After much deliberation, I have decided to use dhtmlXGrid to develop a portal widget for Dataface.  This will improve the usability of Dataface for complex data sources dramatically.  Among other reasons, I have chosen dhtmlXGrid because:

 <ol>   <li>It is available under GPL.</li>     <li>It seems really cool!! and polished.</li>   </ol>     <p>My initial observations are:</p>   <ol>  <li>Documentation and examples seem to be quite good and mature.</li>   <li>There doesn't seem to be a forum or mailing list available.</li>   <li>I am only using the Standard edition.&nbsp; The enterprise edition is not open source.&nbsp;</li>  </ol>   <p>One of the things that I like about this component is that you can load it with XML.&nbsp; This should make it relatively easy to define dataface actions that just output XML that is compatiblle with a the grid.&nbsp; I am stuck in a couple of spots, however.</p> <ol> <li>I can't see how to define como-boxes and select lists using XML.&nbsp; There are examples using javascript to programatically define them (which is okay), but I would rather do everything from XML.</li> <li>.. okay only one thing right now...&nbsp;</li> </ol> <p>I will scour the source code a little bit more and will probably end up emailing the company who makes it to ask about this one.. as it is pretty important.</p><p>The lack of a forum or community provisions seems a little limiting.&nbsp; I feel inclined to set up my own &quot;fan&quot; community site for this, if I begin using it a lot.</p><p>&nbsp;More on this later when I have some results.<br /></p>

Suing Spammers for Fraud

I have recently become more annoyed with spammers and their deceptive tactics.  Especially since I have launched some web sites recently that leaves me open to receive more spam.  I found this interesting article that goes over some of the developments with larger corporations like AOL and Microsoft engaging in court battles with spammers.

Web Lite Translate

My company, Web Lite Solutions, has launched a new web site translation service to translate data-driven websites into multiple languages.

The service uses Dataface as a foundation to convert existing PHP/MySQL web sites into multilingual sites without having to change the site’s structure very much.  Theoretically any web site developed using PHP and MySQL could be converted with minimal changes.  The technology is currently being used to convert Science.ca into French, and it is used to power the http://translate.weblite.ca site itself also.

In the case of Science.ca, attempts had been made to port the site into multilingual frameworks such as Plone or Typo-3 without success.  The site had over 5 years of development invested in the current framework and changing frameworks would have been a lot of work ( = time and money).

Web Lite Translate, on the other hand, was able to cooperate with the existing architecture to seamlessly convert it into a multilingual site.

Check out Web Lite Translate at http://translate.weblite.ca

Source Guardian

This is a follow-up on my experimentation with PHP compilers.  After trying Bcompiler and choosing not to use Zend Guard, I tried out Source Guardian.  It will compile PHP into byte codes for either PHP 4 or PHP5 using a command line executable (or a GUI in Windows).  The PHP file is encoded in place and can be used just as if it wasn’t encoded at all.  You only need to copy a directory of dynamic php extensions into a parent folder of the script (e.g. the web server’s document root would work fine) which are used to decode and run the encoded script.

I tried this out and it worked flawlessly.  Not only that, but this product opens doors to add limited licensing to your products.  You can, for example, set an expiry date on code, or stipulate that the code can only be run on a certain IP or MAC address.

All I can say is that I will definitely be purchasing this product and I recommend it for anyone who distributes PHP source code and doesn’t want to give away all intellectual property. 

Adventures with PHP Compilers

I am in a situation where I need to protect some intellectual property in a PHP script.  The recommended solution is to use a code obfuscator (or compiler) to encode the script.  Currently the landscape leaves much to be desired in this area as far as PHP is concerned.  The industry standard seems to be Zend Guard (http://www.zend.com/products/zend_guard) which is created by Zend – the guys who make PHP – but this has a couple of down sides:

  1. It costs $1000 US.
  2. The web server must be equipped with the Zend Optimizer extension in order to run encoded scripts. AFAIK this extension needs to be compiled statically into PHP.  This would drastically reduce the portability of my scripts. 

The biggest open-source compiler is bcompiler (http://php.net/bcompiler), a PECL extension.  Code that is compiled using this extension can be run with the assistance of the bcompiler extension.  Since bcompiler is a dynamic extension (i.e. it can be loaded at run time) it is theoretically much easier for users to install it (for my script).  Not a perfect solution, but I was willing to accept this hassle in order to ensure the protection of my intellectual property. 

In the past I have had nothing but bad experiences with PECL.  The extensions never install like they are supposed to.  If you make it past the compilation errors and actually get it installed, it is usually an ordeal to turn it on, and can be even more difficult to get it working properly.  These problems are usually related to incompatibilities with versions of gcc, automake, libtool, etc.., and a few hours (or sometimes days) reading through the PECL bugtracker can usually get these problems solved, but, … you get the point.

Unfortunately my experience with bcompiler was no better.   I tried installing it on my laptop running OS X 10.4 and PHP 4.3 (and also PHP 5), but couldn’t get past make.  It gave an undescriptive error "Make failed".  A little bit of googling revealed that bcompiler may be incompatible with GCC 4.x – a bug report suggested I downgrade to version 3.x.  Frankly, I don’t feel like installing an older version of GCC just to get bcompiler to install.

No problem, I don’t need it to run on my lap top.  I have a web host running red hat linux.  Surly it will compile on there.  In fact it did compile.  After about 45 minutes of tinkering with the php.ini file and the bcompiler.so file, I was even able to compile a simple php script into bytecode with the help of bencoder (http://bbs.giga.net.tw/bencoder.php) a script created by Shen Cheng-Da to harness the power of bcompiler.  The only problem is that the compiled code doesn’t work properly.  I can include the compiled script into another script, but cannot call any methods from it – and the script gives a Segmentation Fault on exit without any explanation.

I’ve looked far and wide for a solution to this one.  PECL bug tracker is full of reports of segmentation faults but the solutions don’t appear to transfer easily to this context.

Conclusion:  bcompiler is not ready for primetime yet.  Please someone correct me if I’m wrong on this point.

On to another commercial compiler: Source Guardian.  This is perhaps the most promising of the bunch.  It takes an approach similar to bcompiler in that scripts encoded with this product can be run on any server using a dynamic php extension.  The difference is that this is much more polished and feature rich, and the dynamic php extension is distributed as a binary, so the user doesn’t have to mess around with make and gcc.

I downloaded a 30 trial version of this software and tried to encode one of my scripts.  I am still in the process of trying to get this to work, … I’ll let you all know how it goes.