How to run your Symfony2 PHP app on Azure website

Windows AzureBestComparator is part of the DojoBoost accelerator in Paris and a Bizpark partner. We have the chance to have some partnerships with Microsoft and thus have some support and early access to some technology. Especially about Windows 8 and Azure.

Talking about Azure, they made tremendous upgrades to the platform during the year. They moved from a PAAS platform to a full layered cloud solution. From SAAS to IAAS you can do almost everything. Create a WordPress in a few clicks, create a VM, configure a VPN, create a SQL server, create a mobile service that interface an autogenerated backend storage with your iPhone/Android/Windows phones, scale your services by moving a slider…
Being at the Azure Open Source Summit really amazed me about all the possibilities that are offered to developers and how fast new features comes to the portal. One of those which interested me the most is the possibility to run a PHP website on Azure using the Web Site feature.

About PHP and Azure

PHPYou currently have two main options to run PHP on azure.

First you can simply create a VM using Azure Virtual Machine, after choosing an OS (Windows Server, CentOS, Suse, Ubuntu…) you just connect to your server and configure everything you need. You can change the size of you VM, it goes up to 8 cores and 15GB of RAM. I am running it in production for a few mounts now, and it is pretty stable and reliable. Nevertheless there are some drawbacks. You have to maintain your environment (which is not my job) and it has some limitations in terms of scalability (if you want more power, you will need to maintain another VM and start configuring load balancing and synchronization…). This is pure IAAS.

The other option I recently explored is using Azure website. You simply ask for a computation service which uses a code synchronized via git. It takes care about all the environment maintenance for you and just runs your code. Moreover, you can leverage some nice features such as the possibility to increase the size of the underlying instance (up to 4 cores and 7GB of RAM), but also increase the number of instances (up to 20) seamlessly. Azure takes care of the rest for you. This is more PAAS oriented and let you concentrate on developing your app.

Let’s start a first PHP website:

  1. First of all go to your azure portal
  2. Create a new website using the custom wizard
  3. Select the name of your website, the region, a database (or not) and check the “publish from source control” option
    step2
  4. Choose “git” and “local git repository” as source control setup
    step3
  5. In a third step select your credential, they will be required for git synchronization and ftp login (if you want to)
  6. Now, open the deployments panel in the newly created website you will find there the git url
    step4
  7. On your local machine, simply clone the repository (or set a remote): git clone https://michel@blog-test.scm.azurewebsites.net/blog-test.git
  8. Put your PHP code in the cloned repository, commit, and push to azure
  9. When pushing to azure, it will automatically publish the newly updated code to the website, you can visualize it directly by connecting to the url of the website displayed in the website dashboard

At this point you can upload PHP code to Azure and run it. Here is what you might want to know:

  • You can switch PHP version (5.3 and 5.4) in the configure tab
  • You can scale the size and the number of the instances in the scale tab
  • You can map your domain name using the “manage domain” option available in the bottom bar if you are in “shared” or “reserved” mode (available in the scale tab)
  • If you don’t want to deploy from a local git repo, you can using github, codeplex or bitbuket. It is configurable from the wizard in step 4.
  • You can set environment variables in the configure tab
  • You can choose which branch of your git to deploy in the configure tab
  • You can access the server via ftp using the credential given in step 5 and the ftp configuration in the dashboard
  • You can override the php.ini by giving a .user.ini at the root of your repo

What about Symfony2 websites?

Symfony2 LogoRunning a Symfony2 website is doable but it will require some more work. You will have two main issues to fix:

  1. Activate url rewriting to web/app.php, and reroute web requests to the web directory
  2. Find a way to use the composer and the console

The first issue can be tackled simply by adding a Web.Config file at the root of you repo. It works just like .htaccess for apache but for IIS. Here is the content:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <rewrite>
            <rules>
                <rule name="web directory">
                    <match url=".*" ignoreCase="false" />
                    <action type="Rewrite" url="web/{R:0}" appendQueryString="true" />
                </rule>
                <rule name="app_php handling" stopProcessing="true">
                    <match url="^(.*)$" ignoreCase="false" />
                    <conditions logicalGrouping="MatchAll">
                        <add input="{REQUEST_FILENAME}" matchType="IsFile" ignoreCase="false" negate="true" />
                    </conditions>
                    <action type="Rewrite" url="/web/app.php" appendQueryString="true" />
                </rule>
            </rules>
        </rewrite>
    </system.webServer>
</configuration>

In the Web.Config file we define two rules. The first one reroutes every request to the web directory, and the second one routes the requests to the web/app.php script when there is no physical file requested.

I did not yet tackled the second issue (ie. composer and console use). What I know is that there is no command line access to Azure website but it seems like you can use the exec PHP function from PHP scripts. So everything seems doable with some web interface that simulates command line using PHP functions. I will edit this post as soon as I find a better solution.

What if I want to add some extensions/tweak to PHP runtime?

You may notice that Symfony recommends the use of the intl extension for PHP which is used for internationalization of the validation component (and also in some BCC bundles). You might also need a specific setup of PHP such as a specific version, a configuration which cannot be overridden in the .user.ini file or another extension.

Don’t worry, you can give to Azure you own PHP runtime:

  1. Go to http://windows.php.net/download/ and get the version of PHP you want (which has to be “VC9 x86 Non Thread Safe”)
  2. Unzip it to a directory in your repo, in my case in a ./bin directory
  3. Configure your php.ini file in bin/php.ini
  4. Remove the extensions you don’t want in bin/ext/*
  5. Add, commit and push the files (which might take some time considering the size of your php runtime)
  6. When git is synchronized, go the configure tab on azure
  7. At the bottom of the section, add a handler mapping configuration. The extension is *.php and the script processor path is D:\home\site\wwwroot\bin\php-cgi.exe (because D:\home\site\wwwroot\ is the root of the website and my PHP runtime is in the bin directory)
    step5

Everything is set and you website now runs on your own PHP runtime.

Wrap up

Despite Azure is very young, it offers a lot of possibilities in terms or configuration. I hope this post will help you understand better how it works and how to setup your favorite PHP environment while leveraging all the nice features of Azure.

My take on the Mahout and Myrrix recommendation algorithms

MahoutWhereas BestComparator has his own recommendation engine based on user profiling, behavior analysis and analysis of product specs, I recently wanted to explore the possibilities of the famous recommendation engine built inside Mahout.

First of all, Mahout is a set of machine learning algorithms which leverage the Hadoop environment, providing powerful and scalable algorithms. One of its main target is the recommendation algorithms also known as taste collaborative filtering.

Recommendation algorithms have been made famous by websites such as Amazon, Youtube or Netflix. They use it to make suggestions based on what you bought, watched or liked.

Myrrix

MyrrixOne of the author of Taste/Mahout recommender engine, Sean Owen decided to give the engine a more formal structure by building Myrrix.

Myrrix is a recommendation engine based on Mahout. It offers an out of the box configuration for a recommendation engine accessible with a Rest API. The good to know are:

  • Scalable as Mahout and Hadoop are scalable, using computing parallelization and a distributed file system
  • Runs an optimized version of Taste (currently Taste 3)
  • Runs in real time
  • Can be efficient even with a relatively small amount of data

Recommendation process

The first thing you want to do is to feed your model (ie. Your algorithm) with current observed data. The models aggregates users, items and the associations between them. These associations are called preferences and are qualified by their value, describing the strength of the association between the user and the item.

Feeding the engine means pushing every observed associations with the user id, the item id and the strength. You are simply giving the engine your current taste graph, linking users to items via their tastes.

When your engine is fed, you have to ask it to refresh. Thus it will re-analyze the given graph and compute an actualized, and thus better, model. This may take some time, but Myrrix has the ability to continue answer your requests during this time.

Finally, with your shinny model you can ask questions and get recommendations. Here are the main queries:

  • Recommend to a user
  • Recommend to a group of users
  • Recommend to an anonymous user
  • Recommend similar items
  • Estimate the strength of the preference between an user and an item

With such a panel of tools you can easily guess that answering the question “What item users like me also liked?” becomes accessible.

Consuming Myrrix from PHP

PHPIn order to integrate Myrrix results to my recommendation engine, I had to build a PHP Myrrix client. I decided to use the Guzzle library that provides a really neat way of building a PHP client for Rest APIs. You can download my library on the open source Github Project: https://github.com/michelsalib/bcc-myrrix.

After installing the library, you can write some very fancy code:

// Get a client
$this->client = MyrrixClient::factory(array(
    'hostname' => 'localhost',
    'port'     => 8080,
));

// Put a user/item assocation, here use #101 as an association of strength 0.5 with item #1000
$command = $this->client->getCommand('PostPref', array(
    'userID' => 101,
    'itemID' => 1000,
    'value'  => (string)0.5,
));
$this->client->execute($command);

// Refresh the index
$command = $this->client->getCommand('Refresh');
$this->client->execute($command);

// Get a recommendation for user #101
$command = $this->client->getCommand('GetRecommendation', array(
    'userID' => 101,
));
$recommendation = $this->client->execute($command)->json();

Here we instantiate a Myrrix client hosted on the localhost on port 8080. We put into the model a preference of 0.5 between the user #101 and the item #1000. We then ask the model to refresh. Finally we get a recommendation for the user #101. The recommendation result is an array of item id with their estimated strength for the given user.

The library is pretty straight forward and help you leverage in a very simple way all the powerfulness of the Myrrix engine from PHP.

I also made a Symfony Bundle that helps you get the client from the dependency container, and offers a cleaner configuration process: https://github.com/michelsalib/BCCMyrrixBundle.

Don’t hesitate to get the code, install it and test it. I would be very happy to get contributions, feedbacks or feature requests.

About work, open source and commitment

With this post, I want to take some time to explain my commitment to open source, especially to Symfony and how it is tangled with my everyday work.

What do I do?

I am currently developing for my new startup I founded with two of my associates and friends. We knew each other at engineer school ECE Paris, and were always hungry for entrepreneurship. We worked together for years now, and had some ideas on the way. Some were bad, some were better, and one was genial enough to deserve a real leap into the world of startup creation.

And so we created BestComparator.

BestComparator?

BestComparatorIt is service that helps everyday consumer to choose the best product according to his profile, his urges, and the general quality vs. price value of the products available on the market.

Our ultimate goal is to make the best product recommendation answering this simple sentence: “I am Michel, I am a 23 years old nerd, I need a new Laptop, and most of my time I develop, listen to podcasts and surf on the web.”

We worked really hard, developing the service, creating partnership with price comparison services, and defining exactly how the UX/algorithm would look like. We starting working full time on September 2011. We released an early private preview on October and a public beta on March 2012.

The service is finally online since July 2012 and we already have some nice recommendations and the business is smoothly starting.

This is just a start and many thing could be, and will be, better in the coming months. We also have a heavy roadmap, containing very nice (killer) features, more products, better recommendations, an ecosystem, and some other surprises.

So what do I do in that venture?

BestComparator is a 3 men job. While my associates work on the business and the UX, I spend my time coding the foundations of the service. I am the Data, Architecture, Algorithm, Information System guy. AKA the CTO/Lead developer/Cofounder.

The challenge is quite big. We have to aggregate, consolidate and extract meaning from a gigantic amount of data, from products specifications to our member’s behaviors and desires. Those are coming through various funnel, from our partners, the socials networks, what the users say and don’t say… I needed a solid base with a solid promise.

Three years ago I was already a symfony 1.4 junky. This was with tremendous thrill that I started working with Symfony2 nearly two years ago when the first preview appeared. I saw in Symfony2 a very powerful tool moving at a fast pace, bringing many new core concepts to the PHP world (bundles, annotation, DI…), and gathering a very prolific community. Symfony2 is scalable, fast, flexible and stable. Most of all, it made PHP better with a modern architecture and better practices. Its strength is well known and since we adopted it Drupal, eZ Publish, phpBB and many more followed the way.

As a proof of or commitment you can have a look at the credit page at BestComparator.

As you can see we have a quite exhaustive architecture, composed around Symfony2.1 we are running on Doctrine2.3, ElasticSearch, Behat, Sonata, Assetic, Twig and many more.

I also committed to a community

Of course we started when Symfony2 was young, and in some ways it still is young. Along the way, I was involved in some features of the framework (especially around the translation components), and I pushed into the Sonata project for instance. I also developed some standalone bundles such as the BCCExtraToolsBundle, the BCCCronManagerBundle… which are quite famous now.

Developing such bundles requires time. I love to spend it working on open source projects, especially when it matters to people. Thus, I am very happy today when I can work on BCC Bundles, designed for BestComparator and pushed to the open world.

As a bonus I also develop some C# apps for Windows Phone 7: Gi7 and Readr7. They are open source, and some other might come around the Windows 8 platform.

I try to allocate as much energy that I can, following, debugging and improving these projects. That’s my second commitment.

How about a third commitment

Also I am getting married teaching programming at ECE Paris for a year now. Mostly C and C#, I also have some involvement in some web development and Java courses.

I do it partly because it allow me to earn enough money to pursue my work at BestComparator and mostly because I like to share.

I can also tell you that committing to a hundreds of students is not a so easy commitment and it requires time and patience (and patience too).

Follow up

First of all I enquire to go on BestComparator, things are French for the moment, as long as we need some contracts with local resellers. Here are the common way of getting updates on the venture: twitter, facebook, g+.

You can watch and fork my work on github, and pack your PHP projects on packagist. My windows apps are on the market place.

And if you are interesting in a very prolific and fast growing engineer school (with very talent teachers of course), go have a look on the ECE Paris website.

Symfony2 Logo

Welcome the BCCCronManagerBundle, the Symfony2 bundle that helps you managing your crons

One thing I don’t like to do when maintaining a website, is having to pull out ssh on a daily basis in order to check that everything is running fine. A basic use case is scheduling and watching crons.

So I recently came out with the idea of building a web interface wrapping the use of the crontab command with some tools for watching associated log files.

And thus, the BCCCronManagerBundle was born.

A quick presentation

The BCCCronManagerBundle can already do some nice things:

  • display the cron entries of the cron table, parsing time expression, command, output file, error file and comment
  • guess the last execution time and status
  • display log files
  • add, edit and remove cron entries

The bundle is localized in english and french. The forms also include some shortcuts t easily build common time expression, launch a symfony command or log in the symfony log directory.

The cron list

The cron list

The cron form

The cron form

A cron log file

A cron log file


How it works

Actually, the architecture is quite simple. Everything relies on two classes: CronManager and Cron.

The CronManager launches the [cci]crontab -l[/cci] command in constructor, then extracting each lines in order to build a collection of Cron instance. It has get, add and remove methods in order to access the Cron collection. A raw method build up the cron table string based on the Cron collection and a write method puts it into a temporary file before launching the [cci]crontab $file[/cci] command.

The Cron class is instantiated using the parse static method. Its job is to parse a cron line from the cron table and extract the time expression, command, output, error output and comment (if defined). Based on the output files it can guess if the cron has already been runned (one of the output files is present) and if it was successful or not (the error file is empty). A getExpression method can build the time expression and the _toString is overriden in order to give the cron representation for the cron table.

The interface is quite neat, thanks to Sam. He helped me implementing the twitter bootstrap which is very powerfull and elegant. I also decided to make use of the jQuery plugin that is quite impressive and can easily replace jQueryUI on some points.

Wrap up

You can download and install the bundle on the github : https://github.com/michelsalib/BCCCronManagerBundle.
I also welcome contributions for any improvement, such as a better cron table parsing, more options for cron definition (such as log files), better support of multi platforms, or translations.

Symfony2 Logo

BCCExtraToolsBundle for Symfony2 now includes new features

The purpose of this blog post is to review the last features that have been included into the BCCExtraToolsBundle.
As you may know, its current main feature is a command that extract translation string from your template and dump them into your translation files. It has been so popular that the code has been adapted and merged it into the code Framework a few months ago with the help of the community.
Sadly, I will eventually remove this functionality from the bundle when the 2.1 version will come out. Hopefully, I recently add new cool stuffs into the Bundle recently that you may find useful.

A date parser

You may know that the Bundle currently includes a date formatter that provides you a nice way to localize your dates directly with a twig filter.
I reviewed the code recently to provide a way to make the operation working the other way. You can now do such things :

[cc_php]
get(‘bcc_extra_tools.date_formatter’);

// obtain a datetime instance for a normally formated string
$date = $dateFormatter->parse(‘November 1, 2011′);

// obtain a datetime instance using a defined locale
$date = $dateFormatter->parse(‘1 Novembre 2011, 20;14’, ‘fr’);

// obtains a datetime instance with a uncommon string
$date = $dateFormatter->parse(‘Nov. 2011);

[/cc_php]

Basically, the code is trying every parsing it can using the formats available for the formatter (more information here http://www.michelsalib.com/2011/07/a-twig-extension-that-translates-countries-and-dates/) and more so that it can parse almost anything.
Of course it can be very handy when you consume some weird API using nonstandard datetime formatting.

A unit converter

About consuming API. I can be very annoying to find some values in a unit don’t want to use. The unit formatter is here to help you:

[cc_php]

get(‘bcc_extra_tools.unit_converter’);

// transform a value knowing the source and destination units
echo $unitConverter->convert(1000, ‘m’, ‘km’); // echoes : 1

// transform a value knowing only the destination units
echo $unitConverter->guessConvert(‘1h’, ‘m’); // echoes : 60

[/cc_php]

To do such work, your value will go through different phases::
– A chain unit converter will try to convert using its DI registered unit converters
– A ratio unit converter which converts units that are strictly proportional will take care of most your conversion
– The ratio unit converter supports different kind of units registered using DI, such as length, weight, speed, time… those are called ratio unit providers
– Ratio units providers defines units and ratios related to a specific unit kind. It includes also the locale where the unit is applicable and the prefixes that can be associated to the unit. For instance, the computer capacity unit kind will define the octet, the bit and the byte with their corresponding conversion ratio and the applicable prefixes (k, M, G…).

With such a structure, the converter insure you that any prefix is taken into account, that the conversion is done between coherent units (it won’t try to convert time into money… sadly), and it also takes into account your current locale (the ‘metre’ is the french unit for the ‘meter’).

I’ll try to add more units and documentation in the coming posts.

Wrap up

The BCCExtraToolsBundle continues to provide useful features to the developers. Get the code of the datetime parser and the unit converter on the BCCExtraToolsBundle github : https://github.com/michelsalib/BCCExtraToolsBundle. Don’t hesitate to report issues, feedback and improve the code :)

Symfony2 Logo

Introduce the BCCAutoMapperBundle for Symfony2

One of my favorite tools with my .NET developments is definitely AutoMapper. It allows to easily mapping objects to other objects by generating default maps for graph of objects base on the name of the different members.

I found it quite a shame that such a tool did not exist on the PHP platform. So I started a new Symfony2 bundle to palliate this lack.

How the AutoMapperBundle works

You can find it here: https://github.com/michelsalib/BCCAutoMapperBundle.

As a quick example here to show you how you can map objects together. Here is your model:

[cc_php]
name = $name;
$this->description = $description;
$this->author = new SourceAuthor($author);
}

public getName() {
return $this->name;
}

public getDescription() {
return $this->description;
}

public getAuthor() {
return $this->author;
}
}

class SourceAuthor {
private $name;

public __construct($name) {
$this->name = $name;
}

public getName() {
return $this->name;
}
}

class DestinationPost {
private $title;
private $description;
private $author;

public getTitle() {
return $this->title;
}

public setTitle($title) {
$this->title = title;
}

public getDescription() {
return $this->description;
}

public setDescription($description) {
$this->description = description;
}

public getAuthor() {
return $this->author;
}

public setAuthor($author) {
$this->author = author;
}
}
[/cc_php]

Then, in you application:

[cc_php]
get(‘bcc_auto_mapper.mapper’);
// create default map and route members
$mapper->createMap(‘MySourcePost’, ‘MyDestinationPost’)
->route(‘title’, ‘name’)
->route(‘author’, ‘author.name’);

// create objects
$source = new SourcePost(‘AutoMapper Bundle’, ‘A great bundle’, ‘Michel’);

// map
$destination = $mapper->map($source, $new DestinationPost());

echo $destination->getTitle(); // outputs ‘AutoMapper Bundle’
echo $destination->getDescription(); // outputs ‘A great bundle’
echo $destination->getAuthor(); // outputs ‘Michel’
[/cc_php]

Read the doc on the github to find more explanation and more functionalities: https://github.com/michelsalib/BCCAutoMapperBundle.

This is a very light implementation and some cases are not covered yet (such as dynamic graph creation). Please, don’t hesitate to fork me and provide feedback.

Symfony2 Logo

A twig extension that translates countries and dates

For today’s tutorial, we will see how to create a Twig extension that provides two filters that localize countries and dates.

What is wrong
In Symfony2, you might run into this issue. For instance, countries that are saved by form using the CountryType are stored using a two letter code. For instance you can have US for “United States”, UK for “United Kingdom”… The problem is that you cannot use it directly; you need a read string representation. Moreover, this string representation varies depending on the current locale.
Also you will have exactly the same trouble with dates, which can be “January 1rst” in English and “1er Janvier” in French.

What we need
The easiest way to handle this kind of translation is to use a twig filter. At the end we want this syntax:
– [cci]{{ user.coutnry | country }}[/cci]
– [cci]{{ user.birthday | localeDate }}[/cci]

Note that such a translation cannot be done easily without the use of the apache2 intl module. You have to install it on your server before going any further : http://www.php.net/manual/en/book.intl.php.

Create a twig extension
Adding filters into Symfony2 is quite simple. You need to create a new class that override the [cci]Twig_Extension[/cci] class.

Then we can override the [cci]getFilters[/cci] and the [cci]getName[/cci] functions.
[cc_php]
new Twig_Filter_Function(
‘MyVendorMyBundleTwigTwigExtension::countryFilter’
),
‘localeDate’ => new Twig_Filter_Function(
‘MyVendorMyBundleTwigTwigExtension::localeDateFilter’
),
);
}

public function getName()
{
return ‘myExtensionName';
}
}
[/cc_php]

Note that at this moment the country and the localeDate are defined. When we will use them from a template they will call the defined static functions.

The country filter
The country function will use the list of countries already used by the Symfony2 form component. It is the [cci]SymfonyComponentLocaleLocale[/cci] class:
[cc_php]
<?php
namespace MyVendorMyBundle Twig;

use SymfonyComponentLocaleLocale;

class TwigExtension extends Twig_Extension {

//…

public static function countryFilter($country)
{
$countries = Locale::getDisplayCountries(
Locale::getDefault()
);

return $countries[$country];
}
}
[/cc_php]

The code is pretty straightforward. Just note that to obtain the application locale we use the [cci] Locale::getDefault()[/cci] function, this [cci]Locale[/cci] class is provided by the intl extension. Also when Twig calls the filter, it always sets as first parameter the filtered value.

The localeDate filter
The filter for the date is a little more complicate:
[cc_php]
IntlDateFormatter::NONE,
‘short’ => IntlDateFormatter::SHORT,
‘medium’ => IntlDateFormatter::MEDIUM,
‘long’ => IntlDateFormatter::LONG,
‘full’ => IntlDateFormatter::FULL,
);
$dateFormater = IntlDateFormatter::create(
Locale::getDefault(),
$values[$dateType],
$values[$timeType]
);

return $dateFormater->format($date);
}
}
[/cc_php]

First of all we added two more optional parameters. They will serve to override the kind of rendering we want for the filtered date. We can use them like this :
– [cci]{{ user.createdAt | localeDate(‘long’,’medium’) }}[/cci] for a long date and a medium time representation

Then we create a date formatter which is an instance of [cci]IntlDateFormatter[/cci] parameterized with our format parameters. I used an associative array to translate string parameters to the supported contants.

Then we can simply format the date and return it using the [cci]format[/cci] function.

Activate the twig extension
In order to activate the twig extension, you need to register it as a service to the container. In order for the container to know it is a twig extension, you need to add the [cci]twig.extension [/cci] tag.

Just add to your [cci]config.yml[/cci] file:
[cc]
services:
bcc.twig.extension:
class: MyVendorMyBundleTwigTwigExtension
tags:
– { name: twig.extension }
[/cc]

Wrap up
We build a nice powerful twig extension that provides two filters that helps localize countries and dates.

You simply add the extension to your project by checking out the BCCExtraToolsBundle : https://github.com/michelsalib/BCCExtraToolsBundle.