James Morris - JMOZ http://blog.jmoz.co.uk I pretend I know PHP and internet stuff. Currently contracting at TimeOut London. posterous.com Mon, 12 Dec 2011 13:51:00 -0800 Symfony2 FOSUserBundle Role entities http://blog.jmoz.co.uk/symfony2-fosuserbundle-role-entities http://blog.jmoz.co.uk/symfony2-fosuserbundle-role-entities

I’ve been working with FOSUserBundle Roles recently, adding database persistence using a simple Doctrine array mapping. We needed a better implementation where the Roles could be managed from the database and dynamically added and removed to a User through an admin interface. I found quite a few posts on stackoverflow and the Symfony2 google group asking how best to implement Role entities but very few answers or solutions, and no documentation on the FOSUserBundle github page.

Symfony2’s security system is one of the most complex parts of the framework. Couple this with FOSUserBundle’s weird AOP implementation and you’ve basically got a big massive ball ache when it comes to figuring out what’s going on. After staring through a lot of code and not really getting anywhere, I figured I’d write some tests to cover the existing functionality then refactor the User and Role classes until everything went green.

Unit test to cover existing functionality

For reference here is the unit test I was working with.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
<?php

namespace JMOZ\Bundle\SecurityBundle\Tests\Functional;

use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use JMOZ\Bundle\SecurityBundle\Entity\User;
use JMOZ\Bundle\SecurityBundle\DataFixtures\LoadSecurityData;

/**
* @author James Morris <james@jmoz.co.uk>
*/
class SecurityTest extends WebTestCase
{

    private $_em;
    private $_container;

    /**
* Remember when testing User's they have a default role ROLE_USER which will show +1 in counts
*/
    protected function setUp()
    {
        $kernel = static::createKernel();
        $kernel->boot();
        $this->_container = $kernel->getContainer();
        $this->_em = $this->_container->get( 'doctrine' )->getEntityManager( 'user' );
    }

    private function loadData()
    {
        $data = new LoadSecurityData( $this->_container->get( 'security.encoder_factory' ) );
        $data->truncate( $this->_em );
        $data->load( $this->_em );
    }
    
    private function getRole( $role )
    {
        return $this->_em->getRepository( 'JMOZSecurityBundle:Role' )->findOneBy( array( 'role' => $role) );
    }

    public function testNewUserHasRoleDefaultRole()
    {
        $user = new User();
        $this->assertTrue( $user->hasRole( 'ROLE_USER' ) );
        $this->assertTrue( $user->hasRole( 'ROLE_USER' ) ); // call again in case of the addition of duplicate objects (was a bug)
        $this->assertEquals( 1, count( $user->getRoles() ) );
    }
    
    public function testNewUserNotHasRole()
    {
        $user = new User();
        $this->assertTrue( $user->hasRole( 'ROLE_USER' ) );
        $this->assertFalse( $user->hasRole( 'ROLE_FOO' ) );
    }
    
    public function testNewUserDefaultRoleDoesNotSaveToDb()
    {
        $user = new User();
        $user->setUsername( 'testusertemp' );
        $user->setAlgorithm( 'sha512' );
        $user->setPassword( 'testpass' );
        $user->setEmail( 'testuser1@test.com' );
        
        $this->_em->persist( $user );
        $this->_em->flush();
        $userId = $user->getId();
        $this->_em->clear();
        
        $user = $this->_em->getRepository( 'JMOZSecurityBundle:User' )->find( $userId );
        $this->assertTrue( $user->hasRole( 'ROLE_USER' ) );
        $this->assertEquals( 1, count( $user->getRoles() ) );
        $this->assertEquals( 0, $user->getRolesCollection()->count() ); // the real collection from the db, not the roles array with added default
    }
    
    public function testNewUserAddRoleSaves()
    {
        $this->loadData();

        $user = new User();
        $user->setUsername( 'testusertemp' );
        $user->setAlgorithm( 'sha512' );
        $user->setPassword( 'testpass' );
        $user->setEmail( 'testuser1@test.com' );
        $user->addRole( $this->getRole( 'ROLE_TEST1' ) );
        
        $this->assertEquals( 2, count( $user->getRoles() ) );
        
        $this->_em->persist( $user );
        $this->_em->flush();
        
        $user = $this->_em->getRepository( 'JMOZSecurityBundle:User' )->find( 3 );
        $this->assertTrue( $user->hasRole( 'ROLE_TEST1' ) );
        $this->assertTrue( $user->hasRole( 'ROLE_USER' ) );
        $this->assertEquals( 2, count( $user->getRoles() ) );
    }
    
    public function testNewUserRoleUniqueness()
    {
        $user = new User();
        $user->addRole( $this->getRole( 'ROLE_TEST1' ) );
        $user->addRole( $this->getRole( 'ROLE_TEST2' ) );
        $user->addRole( $this->getRole( 'ROLE_TEST2' ) );
        $user->addRole( $this->getRole( 'ROLE_TEST2' ) );
        $this->assertEquals( 3, count( $user->getRoles() ) );
    }
    
    public function testNewUserRemoveRole()
    {
        $user = new User();
        $user->addRole( $this->getRole( 'ROLE_TEST1' ) );
        $user->addRole( $this->getRole( 'ROLE_TEST2' ) );
        $this->assertEquals( 3, count( $user->getRoles() ) );
        $user->removeRole( 'ROLE_TEST1' );
        $user->removeRole( 'ROLE_TEST2' );
        $this->assertEquals( 1, count( $user->getRoles() ) );
    }
    
    public function testExistingUserRemoveRoleSaves()
    {
        $this->loadData();

        $user = $this->_em->getRepository( 'JMOZSecurityBundle:User' )->find( 1 );
        $this->assertEquals( 3, count( $user->getRoles() ) );
        $user->removeRole( 'ROLE_TEST2' );
        $this->_em->flush();
        $this->_em->clear();
        $user = $this->_em->getRepository( 'JMOZSecurityBundle:User' )->find( 1 );
        $this->assertEquals( 2, count( $user->getRoles() ) );
    }
    
    public function testNewUserSetRoles()
    {
        $user = new User();
        $user->addRole( $this->getRole( 'ROLE_TEST1' ) );
        $user->addRole( $this->getRole( 'ROLE_TEST2' ) );
        $this->assertEquals( 3, count( $user->getRoles() ) );
        $this->assertTrue( $user->hasRole( 'ROLE_TEST1' ) );
        $this->assertTrue( $user->hasRole( 'ROLE_TEST2' ) );
        
        $user->setRoles( array( $this->getRole( 'ROLE_TAXONOMY_VIEW' ), $this->getRole( 'ROLE_TEST2' ) ) );
        $this->assertEquals( 3, count( $user->getRoles() ) );
        $this->assertTrue( $user->hasRole( 'ROLE_TAXONOMY_VIEW' ) );
        $this->assertTrue( $user->hasRole( 'ROLE_TEST2' ) );
        $this->assertFalse( $user->hasRole( 'ROLE_TEST1' ) );
    }

    public function testExistingUserWithRoles()
    {
        $this->loadData();

        $user = $this->_em->getRepository( 'JMOZSecurityBundle:User' )->find( 1 );
        $this->assertTrue( $user->hasRole( 'ROLE_TEST1' ) );
        $this->assertTrue( $user->hasRole( 'ROLE_TEST2' ) );
        $this->assertTrue( $user->hasRole( 'ROLE_USER' ) );
        $this->assertFalse( $user->hasRole( 'ROLE_FOO' ) );
        $this->assertEquals( 3, count( $user->getRoles() ) );
    }

    /**
* Login the testuser (from fixture data) into the system.
* The login proxess will trigger a lot of the User and Role code so is a useful test.
* @return $client
*/
    private function login( $username = 'testuser', $password = 'testpass' )
    {
        $client = self::createClient();
        $crawler = $client->request( 'GET', '/login' );
        $form = $crawler->selectButton( 'Login' )->form( array( '_username' => $username, '_password' => $password ) );
        $client->submit( $form );
        $this->assertEquals( 302, $client->getResponse()->getStatusCode() );
        $client->followRedirect();
        $this->assertEquals( 200, $client->getResponse()->getStatusCode() );
        return $client;
    }

    /**
* testuser does not have ROLE_TAXONOMY_VIEW which /taxonomy is locked down with
*/
    public function testTaxonomyRoleTaxonomyView()
    {
        $client = $this->login();
        $client->request( 'GET', '/taxonomy/' );
        $this->assertEquals( 403, $client->getResponse()->getStatusCode() );
    }

    /**
* testuser2 has ROLE_TAXONOMY_VIEW
*/
    public function testTaxonomyRoleTaxonomyViewOk()
    {
        $client = $this->login( 'testuser2', 'testpass' );
        $client->request( 'GET', '/taxonomy/' );
        $this->assertEquals( 200, $client->getResponse()->getStatusCode() );
    }
    
}

Creating the Role entity

FOSUserBundle provides no Role entity. Symfony2 provides a Role object but some class members are private so we just implement the Role interface, hoping it will ensure the new Role implementation works correctly with the security system.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
<?php

namespace JMOZ\Bundle\SecurityBundle\Entity;

use Doctrine\ORM\Mapping as ORM;
use Symfony\Component\Security\Core\Role\RoleInterface;

/**
* Role Entity
*
* @ORM\Entity
* @ORM\Table( name="security_roles" )
*
* @author James Morris <james@jmoz.co.uk>
* @package SecurityBundle
*/
class Role implements RoleInterface
{

    /**
* @ORM\Id
* @ORM\Column(type="integer", name="id")
* @ORM\GeneratedValue(strategy="AUTO")
*/
    private $id;

    /**
* @ORM\Column(type="string", name="role", unique="true", length="70")
*/
    private $role;

    /**
* Populate the role field
* @param string $role ROLE_FOO etc
*/
    public function __construct( $role )
    {
        $this->role = $role;
    }

    /**
* Return the role field.
* @return string
*/
    public function getRole()
    {
        return $this->role;
    }

    /**
* Return the role field.
* @return string
*/
    public function __toString()
    {
        return (string) $this->role;
    }

}

It’s a simple object, I’m implementing a __toString() method so we can loop in the template over User::getRoles() and echo the $role.

Creating the User, Role relationship

This is the User class with the Role relationship mapped. I tried to implement the same Role functionality as FOSUserBundle. You are restricted to certain method parameters due to the type hinting in the parent class, e.g. setRoles() must take an array. I found type hinting and return type expectations in some of the symfony2 security layer code, such as:

UsernamePasswordToken::__construct($user, $credentials, $providerKey, array $roles = array())

Because of this, I mixed up an ArrayCollection and array implementation. You can see I also provided the (set|get)RolesCollection() methods to make things easier when working with doctrine.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
<?php

namespace JMOZ\Bundle\SecurityBundle\Entity;

use Doctrine\ORM\Mapping as ORM;
use Doctrine\Common\Collections\ArrayCollection;
use Doctrine\Common\Collections\Collection;
use FOS\UserBundle\Model\User as BaseUser;

/**
* User Entity
*
* @ORM\Entity( repositoryClass="JMOZ\Bundle\SecurityBundle\Entity\Repository\UserRepository" )
* @ORM\Table( name="security_users" )
*
* @author James Morris <james@jmoz.co.uk>
* @package SecurityBundle
*/
class User extends BaseUser
{
    //... existing class members

    /**
* @ORM\ManyToMany(targetEntity="Role")
* @ORM\JoinTable(name="security_users_roles")
*/
    protected $roles;
    
    public function __construct()
    {
        parent::__construct();
        $this->roles = new ArrayCollection();
    }
    
   //... existing setters/getters
    
    /**
* Returns an ARRAY of Role objects with the default Role object appended.
* @return array
*/
    public function getRoles()
    {
        return array_merge( $this->roles->toArray(), array( new Role( parent::ROLE_DEFAULT ) ) );
    }
    
    /**
* Returns the true ArrayCollection of Roles.
* @return Doctrine\Common\Collections\ArrayCollection
*/
    public function getRolesCollection()
    {
        return $this->roles;
    }
    
    /**
* Pass a string, get the desired Role object or null.
* @param string $role
* @return Role|null
*/
    public function getRole( $role )
    {
        foreach ( $this->getRoles() as $roleItem )
        {
            if ( $role == $roleItem->getRole() )
            {
                return $roleItem;
            }
        }
        return null;
    }
    
    /**
* Pass a string, checks if we have that Role. Same functionality as getRole() except returns a real boolean.
* @param string $role
* @return boolean
*/
    public function hasRole( $role )
    {
        if ( $this->getRole( $role ) )
        {
            return true;
        }
        return false;
    }

    /**
* Adds a Role OBJECT to the ArrayCollection. Can't type hint due to interface so throws Exception.
* @throws Exception
* @param Role $role
*/
    public function addRole( $role )
    {
        if ( !$role instanceof Role )
        {
            throw new \Exception( "addRole takes a Role object as the parameter" );
        }
        
        if ( !$this->hasRole( $role->getRole() ) )
        {
            $this->roles->add( $role );
        }
    }
    
    /**
* Pass a string, remove the Role object from collection.
* @param string $role
*/
    public function removeRole( $role )
    {
        $roleElement = $this->getRole( $role );
        if ( $roleElement )
        {
            $this->roles->removeElement( $roleElement );
        }
    }
    
    /**
* Pass an ARRAY of Role objects and will clear the collection and re-set it with new Roles.
* Type hinted array due to interface.
* @param array $roles Of Role objects.
*/
    public function setRoles( array $roles )
    {
        $this->roles->clear();
        foreach ( $roles as $role )
        {
            $this->addRole( $role );
        }
    }
    
    /**
* Directly set the ArrayCollection of Roles. Type hinted as Collection which is the parent of (Array|Persistent)Collection.
* @param Doctrine\Common\Collections\Collection $role
*/
    public function setRolesCollection( Collection $collection )
    {
        $this->roles = $collection;
    }
}

Modify the schema

Check the changes to the database:

$ php app/console --env=test doctrine:schema:update --em=user --dump-sql
CREATE TABLE security_roles (id INT AUTO_INCREMENT NOT NULL, role VARCHAR(70) NOT NULL, UNIQUE INDEX UNIQ_5A82CD6D57698A6A (role), PRIMARY KEY(id)) ENGINE = InnoDB;
CREATE TABLE security_users_roles (user_id INT NOT NULL, role_id INT NOT NULL, INDEX IDX_71E6DDEFA76ED395 (user_id), INDEX IDX_71E6DDEFD60322AC (role_id), PRIMARY KEY(user_id, role_id)) ENGINE = InnoDB;
ALTER TABLE security_users_roles ADD CONSTRAINT FK_71E6DDEFA76ED395 FOREIGN KEY (user_id) REFERENCES security_users(id) ON DELETE CASCADE;
ALTER TABLE security_users_roles ADD CONSTRAINT FK_71E6DDEFD60322AC FOREIGN KEY (role_id) REFERENCES security_roles(id) ON DELETE CASCADE;
ALTER TABLE security_users DROP roles

Either run this using the --force parameter or create a migration.

Issues

This will break the FOSUserBundle promote and demote commands as they are hard coded to use addRole(string).

Summary

I managed to get all the tests to pass:

$ phpunit --stop-on-error --stop-on-failure -c app src/JMOZ/Bundle/SecurityBundle/Tests/Functional/SecurityTest.php
PHPUnit 3.5.5 by Sebastian Bergmann.

...........

Time: 30 seconds, Memory: 82.25Mb

OK (11 tests, 35 assertions)

I would rather have not mixed the array/ArrayCollection implementation but was forced to do so due to the existing functionality and interfaces. I’d have liked to have seen some other implementations or solutions but could not seem to find any. If anyone has seen anything better please let me know. I hope this helps someone out there!

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Mon, 05 Dec 2011 02:43:47 -0800 How to find out your Ubuntu version http://blog.jmoz.co.uk/find-out-ubuntu-version http://blog.jmoz.co.uk/find-out-ubuntu-version

To find out your Ubuntu version (or codename) use the lsb_release command which will “print distribution-specific information”:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 11.04
Release:        11.04
Codename:       natty

Or if you just want the string of the codename:

$ lsb_release -s -c
natty

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Wed, 30 Nov 2011 10:49:00 -0800 Symfony2 Warning: session_start(): The session id is too long or contains illegal characters... http://blog.jmoz.co.uk/symfony2-warning-sessionstart-the-session-id http://blog.jmoz.co.uk/symfony2-warning-sessionstart-the-session-id

Whilst trying to set up a new test environment I’ve run into another lovely Symfony2 bug/error. I have a functional test with a body as such:

public function testFoo()
{
    $client = self::createClient();
    $client->request('GET', '/foo/');
    $this->assertEquals( 403, $client->getResponse()->getStatusCode() );
}

FYI I’m trying to test access roles are working correctly so I should get a 403 forbidden.

And my relevant config_test.yml:

imports:
    - { resource: config_dev.yml }
    - { resource: parameters_test.ini }

framework:
    router:   { resource: "%kernel.root_dir%/config/routing_test.yml" }
    test:     ~

Upon running this test I get the lovely error:

Warning: session_start(): The session id is too long or contains illegal characters, valid characters are a-z, A-Z, 0-9 and '-,' in /home/james/foo/vendor/symfony/src/Symfony/Component/HttpFoundation/SessionStorage/NativeSessionStorage.php line 87 (500 Internal Server Error)

After messing with the config and looking around, it turns out this is a known issue. For whatever reason, my config was missing some required session parameters for testing. Have a look at https://github.com/symfony/symfony/issues/1759 for more info. To fix it, I had to change the session storage method params in config_test.yml from native (inherited) to filesystem:

framework:
    router:   { resource: "%kernel.root_dir%/config/routing_test.yml" }
    test:     ~
    session:
      storage_id: session.storage.filesystem

Now, the exception is gone and the test runs:

$ phpunit --stop-on-error --stop-on-failure -c app src/Foo/Bundle/SecurityBundle/
PHPUnit 3.5.5 by Sebastian Bergmann.

..F

Time: 1 second, Memory: 29.75Mb

There was 1 failure:

1) Foo\Bundle\FooBundle\Tests\Functional\SecurityTest::testFoo
Failed asserting that  matches expected .

/home/james/code/Foo/Bundle/SecurityBundle/Tests/Functional/SecurityTest.php:59

FAILURES!
Tests: 3, Assertions: 4, Failures: 1.

Great, next problem…

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Wed, 23 Nov 2011 02:30:00 -0800 Symfony2 FOSUserBundle Roles - a simple solution http://blog.jmoz.co.uk/symfony2-fosuserbundle-roles http://blog.jmoz.co.uk/symfony2-fosuserbundle-roles

Out of the box, FOSUserBundle does NOT support Doctrine ORM database persisted Roles. The base class they give you to extend has functionality for roles, i.e. setting, getting and checking them, but you will need to write your own code to persist to the database and thus fully equipping the User with role functionality.

The quickest and easiest way to get roles up and running is to use the existing FOS functionality and create your user a new roles field, mapping it as an array type in Doctrine. This is quicker and easier to work with than using separate Role objects. So your user class will need to specify the $roles class member with the following mapping metadata:

/**
 * @ORM\Column(type="array", name="roles")
 */
 protected $roles;

This will tell doctrine to map a PHP array to a SQL CLOB using serialize() and unserialize(). Take a look at the Doctrine mapping documentation at http://www.doctrine-project.org/docs/orm/2.0/en/reference/basic-mapping.html#...

Schema changes

So now the model has been sorted, the new column needs to be added to the database. Generate a migration for the changes to the users table:

$ php app/console doctrine:migrations:diff
Generated new migration class to "/home/james/fooapp/DoctrineMigrations/Version20111121145741.php" from schema differences.

This will generate said file which will contain a call to a method to add the new column:

$this->addSql("ALTER TABLE users ADD roles LONGTEXT NOT NULL COMMENT '(DC2Type:array)'");

If you want to set up default roles for existing users, under the call to addSql() insert the following line:

$this->addSql(sprintf("UPDATE users SET roles = '%s'", 'a:1:{i:0;s:8:"ROLE_FOO";}'));

Then execute:

$ php app/console doctrine:migrations:migrate
...
Migrating up to 20111121145921 from 0

  ++ migrating 20111121145921

     -> ALTER TABLE users ADD roles LONGTEXT NOT NULL COMMENT '(DC2Type:array)'
     -> UPDATE users SET roles = 'a:1:{i:0;s:8:"ROLE_FOO";}'

  ++ migrated (0.69s)

  ------------------------

  ++ finished in 0.69
  ++ 1 migrations executed
  ++ 2 sql queries

(More information on migrations can be found at http://symfony.com/doc/2.0/bundles/DoctrineMigrationsBundle/index.html and https://github.com/symfony/DoctrineMigrationsBundle)

This will add the new column to your users table. You now need to create a new user which will save a serialized empty array in the roles column. From here you can add new roles to that user or you may need to sort out existing users which will have nothing in their roles field unless you followed the step above. Trying to update or work with existing users with empty roles fields will most likely give you a Doctrine ConversionException:

[Doctrine\DBAL\Types\ConversionException]                   
Could not convert database value "" to Doctrine Type array

So I suggest setting up a new user with default roles for your system then copying the content of that user’s roles field into existing user’s roles fields.

To test the roles functionality try the following:

$ php app/console fos:user:create jamestest james@foo.co.uk mypassword
Created user jamestest
$ php app/console fos:user:promote jamestest --super
User "jamestest" has been promoted as a super administrator.

If you now check your db, the roles field for jamestest should look similar to:

a:2:{i:0;s:8:"ROLE_FOO";i:1;s:16:"ROLE_SUPER_ADMIN";}

Your User should now be set up to persist it’s roles to the database. You should be able to edit the security.yml file and restrict access to certain parts of your site by url pattern:

access_control:
  - { path: ^/admin, roles: ROLE_FOO }

For further information see http://symfony.com/doc/2.0/book/security.html#securing-specific-url-patterns

UPDATE:

See my following article if you need more powerful Role entity based solution http://blog.jmoz.co.uk/symfony2-fosuserbundle-role-entities

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Wed, 07 Sep 2011 02:00:00 -0700 PHP news http://blog.jmoz.co.uk/php-news http://blog.jmoz.co.uk/php-news

I've noticed recently a lack of PHP news/articles/blog posts coming to my attention.  I'm sure I used to see a constant stream of them but I can't remember my sources.  Maybe its my increasing reliance on Twitter as a source of everything - general news, local info, tech news and banter with friends.  I used to use Google Reader a lot, and sites such as Digg, Popurls, Mixx etc but they all seem to have dropped in quality or they get increasingly harder to manage (GReader rss feeds) whilst everywhere else seems to have a Twitter feed.  So this is more of a request for comments or tips on decent sources of PHP news.  You can reply by comment or tweet me at @jwm0z.  I'll list some of mine:

Twitter

Any tech site or person that tweets about PHP I tend to follow, I have a twitter tech list (which probably could do with being split into a php list) and I also watch a #php and #symfony2 search in my Tweetdeck.

Web Sites

One of my favourite sites to follow is PHP Developer, they reblog a lot PHP articles from developer blogs (including mine) and they have a busy twitter feed.

Some others include Planet PHP, DZone PHP, Zend Developer Zone but I never seem to catch their updates as often as I do as PHP Developer's.

Too many channels

There's just too many channels of information available to be able to get a good overview of the PHP scene.  For me, Twitter is probably the easiest to manage but I need to follow more decent sources of technical articles and news.  If you've got any lists to share or think it's worthwhile me following you, please get in touch!

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Mon, 05 Sep 2011 01:00:00 -0700 A Symfony2 console Command and the Foursquare API venuehistory http://blog.jmoz.co.uk/symfony2-command-foursquare-api http://blog.jmoz.co.uk/symfony2-command-foursquare-api

I've been playing with the Foursquare API recently, I'm attempting to get a new homepage built and want to display a map of where I hang out. I use Foursquare quite a bit so wanted to get the locations from their API then plot them on Google maps.

There's a venuehistory endpoint that gives you a massive list of all the venues you've checked into with details such as the venue name, location including lat and lng coordinates, and the count of times you've checked in.

Here's a screenshot of their json response.

Media_httpdldropboxco_gdpci

I want to pull this into my own database so I have a local copy of it and can play around with it. I'm using Symfony2 so I've created a console command which I can run on a cron, or on demand. I decided to do it this way as mainly a learning exercise and also this is a bit better than a standard php script.

The command hits the endpoint with my Oauth access token (outside of the scope of this article, but see the docs for details) and then simply loops over the response and creates a load of Checkin entities which the entity manager then persists.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
<?php

namespace JMOZ\FoursquareBundle\Command;

use Symfony\Bundle\FrameworkBundle\Command\ContainerAwareCommand;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use JMOZ\FoursquareBundle\Entity\Checkin;

/**
 * Pull 4sq venuehistory into checkins
 * @author James Morris <james@jmoz.co.uk>
 */
class CheckinsCommand extends ContainerAwareCommand {

    protected function configure() {
        $this->setName('4sq:checkins')
                ->setDescription('Pull venue history from Foursquare into checkins table')
                ->addArgument('access_token', InputArgument::OPTIONAL, 'oauth access_token (optional, defaults to 4sq.access_token parameter)')
                ->addOption('no-truncate', $shortcut = null, InputOption::VALUE_NONE, 'Skip TRUNCATE table')
                ->setHelp(<<<EOT
The <info>4sq:checkins</info> command hits the 4sq venuehistory endpoint and populates
the checkins table.
EOT
        );
    }

    protected function execute(InputInterface $input, OutputInterface $output) {
        $access_token = $input->getArgument('access_token') ?: $this->getContainer()->getParameter('4sq.access_token');

        $response = @file_get_contents(sprintf('https://api.foursquare.com/v2/users/%s/venuehistory?oauth_token=%s', $this->getContainer()->getParameter('4sq.my_user_id'), $access_token));
        if (!$response) {
            throw new \Exception("Error requesting venuehistory");
        }
        $responseDecoded = json_decode($response, true);

        $em = $this->getContainer()->get('doctrine')->getEntityManager();

        // (optionally) truncate table first
        if (!$input->getOption('no-truncate')) {
            $em->getConnection()->exec($em->getConnection()->getDatabasePlatform()->getTruncateTableSql('checkins'));
        }

        foreach ($responseDecoded['response']['venues']['items'] as $key => $item) {
            $checkin = new Checkin();
            $checkin->setCount($item['beenHere']);
            $checkin->setVenue($item['venue']['name']);
            $checkin->setLat($item['venue']['location']['lat']);
            $checkin->setLng($item['venue']['location']['lng']);
            $em->persist($checkin);
        }

        $em->flush();
    }
}

The command takes an optional argument for the access_token or otherwise defaults to a parameter. There's also an optional flag to disable the truncation of the checkins table. You can see how to set these up along with the help text in the configure() method. Output is handled by the call to $output->writeln() and you can see how highlighting works in the screenshot of the output.

Media_httpdldropboxco_rkcdq

I may write a future post showing how to push the Foursquare code into a service based implementation so look out.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Wed, 24 Aug 2011 13:26:33 -0700 How to strip exif data using Imagick http://blog.jmoz.co.uk/imagick-strip-exif-data http://blog.jmoz.co.uk/imagick-strip-exif-data

Today I spent a good amount of time trying to figure out how to strip exif data from an image using Imagick.  The first port of call was the (pathetic) documentation at php.net.  I searched for 'exif' but found nothing.  Google isn't too helpful either.  There's a method Imagick::setImageProperty() but that didn't seem to save the data correctly.  There's also an exif extension but it only reads the exif data, not writes it.

I was tipped off by a colleague to the method Imagick::stripImage() which apparently did what I wanted.  The only mention of exif is in a rather helpful comment at the bottom of the page.

To remove the exif data you need to call Imagick::stripImage() on the populated Imagick object and then call Imagick::writeImage() to save your changes.  I've written a test script below that uses a sample image with exif data from the exif website.

Media_httpwwwexiforgs_hijai

Here's the script:

1
2
3
4
5
6
7
8
9
10
11
<?php

$file = '/home/moz/Desktop/canon-ixus.jpg';

$i = new Imagick($file);
print_r($i->getImageProperties('exif:*'));
$i->stripImage();
$i->writeImage($file . '2');

$i->readImage($file . '2');
print_r($i->getImageProperties('exif:*'));

And here's the output:

moz on mz-904 in ~ 
 $ php imagick.php
Array
(   
    [exif:ApertureValue] => 262144/65536
    [exif:ColorSpace] => 1
    [exif:ComponentsConfiguration] => 1, 2, 3, 0
    [exif:CompressedBitsPerPixel] => 3/1
    [exif:Compression] => 6
    [exif:DateTime] => 2001:06:09 15:17:32
    [exif:DateTimeDigitized] => 2001:06:09 15:17:32
    [exif:DateTimeOriginal] => 2001:06:09 15:17:32
    [exif:ExifImageLength] => 480
    [exif:ExifImageWidth] => 640
    [exif:ExifOffset] => 184
    [exif:ExifVersion] => 48, 50, 49, 48
    [exif:ExposureBiasValue] => 0/3
    [exif:ExposureTime] => 1/350
    [exif:FileSource] => 3
    [exif:Flash] => 0
    [exif:FlashPixVersion] => 48, 49, 48, 48
    [exif:FNumber] => 40/10
    [exif:FocalLength] => 346/32
    [exif:FocalPlaneResolutionUnit] => 2
    [exif:FocalPlaneXResolution] => 640000/206
    [exif:FocalPlaneYResolution] => 480000/155
    [exif:InteroperabilityIndex] => R98
    [exif:InteroperabilityOffset] => 1088
    [exif:InteroperabilityVersion] => 48, 49, 48, 48
    [exif:JPEGInterchangeFormat] => 1524
    [exif:JPEGInterchangeFormatLength] => 5342
    [exif:Make] => Canon
    [exif:MakerNote] => 10, 0, 1, 0, 3, 0, 19, 0, 0, 0, 120, 3, 0, 0, 2, 0, 3, 0, 4, 0, 0, 0, 158, 3, 0, 0, 3, 0, 3, 0, 4, 0, 0, 0, 166, 3, 0, 0, 4, 0, 3, 0, 15, 0, 0, 0, 174, 3, 0, 0, 0, 0, 3, 0, 6, 0, 0, 0, 204, 3, 0, 0, 6, 0, 2, 0, 32, 0, 0, 0, 216, 3, 0, 0, 7, 0, 2, 0, 24, 0, 0, 0, 248, 3, 0, 0, 8, 0, 4, 0, 1, 0, 0, 0, 243, 105, 15, 0, 9, 0, 2, 0, 32, 0, 0, 0, 16, 4, 0, 0, 16, 0, 4, 0, 1, 0, 0, 0, 0, 0, 4, 6, 0, 0, 0, 0, 38, 0, 2, 0, 0, 0, 3, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 2, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 2, 0, 90, 1, 211, 0, 158, 0, 0, 0, 0, 0, 0, 0, 0, 0, 30, 0, 0, 0, 140, 0, 2, 1, 128, 0, 14, 1, 0, 0, 0, 0, 0, 0, 1, 0, 4, 0, 0, 0, 0, 0, 0, 0, 2, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 73, 77, 71, 58, 74, 80, 69, 71, 32, 102, 105, 108, 101, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 70, 105, 114, 109, 119, 97, 114, 101, 32, 86, 101, 114, 115, 105, 111, 110, 32, 49, 46, 48, 0, 0, 0, 0, 84, 111, 109, 32, 82, 111, 119, 97, 110, 32, 97, 110, 100, 32, 83, 97, 114, 97, 104, 32, 67, 108, 105, 102, 116, 111, 110, 0, 0, 0, 0, 0
    [exif:MaxApertureValue] => 194698/65536
    [exif:MeteringMode] => 2
    [exif:Model] => Canon DIGITAL IXUS
    [exif:Orientation] => 1
    [exif:RelatedImageLength] => 640
    [exif:RelatedImageWidth] => 480
    [exif:ResolutionUnit] => 2
    [exif:SensingMethod] => 2
    [exif:ShutterSpeedValue] => 553859/65536
    [exif:SubjectDistance] => 3750/1000
    [exif:UserComment] => 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
    [exif:XResolution] => 180/1
    [exif:YCbCrPositioning] => 1
    [exif:YResolution] => 180/1
)
*** stripImage()
Array
(
)

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Wed, 24 Aug 2011 02:26:41 -0700 Nginx 301 to external url http://blog.jmoz.co.uk/nginx-301-to-external-url http://blog.jmoz.co.uk/nginx-301-to-external-url

I'm currently migrating my posterous blog domain name vietnameseinshoreditch.co.uk to something which I think is a little better and more SEO friendly, shoreditchvietnamese.co.uk.  To redirect the existing urls on the old domain to the new domain I need to set up an Nginx 301 redirect.

Currently my DNS A records for vietnameseinshoreditch.co.uk point to posterous' IP.  I want to point them to my own server which is running Nginx and get Nginx to 301 any requests to my new domain (which will point to posterous).  This way any previous links lying around will still work and I'll maintain all of Google's link juice and eventually shoreditchvietnamese.co.uk will be seen as the canonical url.

Here's the Nginx config:

1
2
3
4
5
server {
    listen 80;
    server_name vietnameseinshoreditch.co.uk;
    rewrite ^ http://shoreditchvietnamese.co.uk$request_uri? permanent;
}

Simple.  This 301s all requests to the new domain name and maintains the full request uri.

Give it a go, the link below should get 301'd from vietnameseinshoreditch.co.uk to shoreditchvietnamese.co.uk.

http://vietnameseinshoreditch.co.uk/welcome-to-vietnamese-in-shoreditch

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Thu, 18 Aug 2011 02:35:00 -0700 Mixcloud Downloader is on Twitter! http://blog.jmoz.co.uk/mixcloud-downloader-is-on-twitter http://blog.jmoz.co.uk/mixcloud-downloader-is-on-twitter

When somebody tries to download something off Mixcloud Downloader, it now posts to Twitter.

Just follow @clouddownload for the updates.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Wed, 27 Jul 2011 03:00:00 -0700 Vietnamese in Shoreditch http://blog.jmoz.co.uk/vietnamese-in-shoreditch http://blog.jmoz.co.uk/vietnamese-in-shoreditch

I've launched a new blog, Vietnamese in Shoreditch - the clue's in the name!

I live in Shoreditch.  I eat a shit load of Vietnamese.  I've been to Tay Do Cafe over 30 times.  If i'd have written a blog post every time i'd eaten Vietnamese, i'd have a shit load of pretty sweet content.  Unfortunately I didn't, but things are now going to change, at least I hope they will for a while until I get bored or think of a new fad to focus on.

It's going to be the #1 destination for all information relating to Vietnamese in the Shoreditch area.  Well it may be, we'll see.  Follow @shoreditchviet for post updates.

Vietnamese in Shoreditch can be found at http://vietnameseinshoreditch.co.uk.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Tue, 17 May 2011 04:41:00 -0700 SplObserver, SplSubject - Removing dependencies with the Observer pattern http://blog.jmoz.co.uk/removing-dependencies-with-the-observer-patte http://blog.jmoz.co.uk/removing-dependencies-with-the-observer-patte

I work with symfony every day – I’m a backend developer who mainly enjoys designing and writing domain objects, core services and APIs.  I like to create loosely coupled objects where each object has a clear role and is composed of other objects.  Working on a symfony app, you usually have a mix of domain objects that are used by symfony actions, interspersed with symfony specific code such as logging and sfContext type stuff.  A common bad practice I see is symfony specific code peppered inside of domain objects that could be used elsewhere (such as inside of a Zend app or a script from the cli) but now can’t as they’re coupled to the symfony code.

The Observer pattern

One way you can remove an unwanted dependency is to use the Observer pattern – the dependency is pushed from inside the subject object to the client code that initialises the subject. The subject object exposes events in the code (such as before or after certain method calls) by calling a method notify(), this then notifies all observers that were set up at run time. The subject passes an instance of itself when each observer is notified so that whatever the implementation of the observer, it has full access to the context of when notify() was called. In (somewhat) simple terms, you pull out the line of code that is a dependancy, replace it with a call to notify(), then push the dependant code into an object that implements update() and attach this observer at runtime, we will use the interfaces provided by SPL – the Standard PHP Library, SplSubject and SplObserver.

The Scenario

Right, so I’ve tried to articulate this but as always a code example is best.  The scenario is as follows and is something I’ve seen numerous times: we have a symfony action that hits a web service and for this example just spits out the response.  There are Service classes (which I won’t show an implementation for as they’re simple) which basically contain some business logic, parameters, and endpoint details such as url and HTTP method.  The Client object takes a Service object and transport object (HTTP in this case) puts them both together and sends a HTTP request to the web service endpoint and returns the response.  Inside all of this is the need for logging so there is symfony specific code coupled to the cohesive and loosely coupled Client object. I’m going to show some example code and then refactor it till eventually the dependency is removed and the Observer pattern is implemented.  Obviously there are many ways to do this and many reasons for and against it.  Do not take this as dogma – it’s merely here to demonstrate an option you can take.  And none of this code is tested, just free styled.

The Action

The action simply chucks a Service and Http object into the client for it to use and calls it.

1
2
3
4
5
6
<?php

public function executeWebServiceCall() {
    $client = new ServiceClient(new FooService(), new Http());
    $this->renderText($client->call());
}

The Client

The ServiceClient takes an instance of a Service and a Http object which are injected at construction.  There’s an accessor and a mutator so the Service can be switched at runtime.  The main code is inside call() which sets up the Http request and returns the response.  You can also see we are logging the Service call details by getting the logger from sfContext - not nice.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
<?php

class ServiceClient {
    
    private $service;
    private $http;
    
    public function __construct(Service $service, Http $http) {
        $this->service = $service;
        $this->http = $http;
    }

    public function getService() {
        return $this->service;
    }

    public function setService(Service $service) {
        $this->service = $service;
    }
    
    public function call() {
        sfContext::getInstance()->getLogger()->info("Calling service " . get_class($this->service) . " with url {$this->service->getUrl()}");
        $this->http->setUrl($this->service->getUrl());
        $this->http->setMethod($this->service->getMethod());
        $this->http->setBody($this->service->getBody());
        $this->http->send();
        return $this->http->getResponseBody();
    }
}

Refactoring out sfContext

Nobody likes sfContext calls inside domain objects.  Fact.  So let’s make use of dependency injection and push it to the client code.  Here’s the new action and Client.

1
2
3
4
5
6
<?php

public function executeWebServiceCall() {
    $client = new ServiceClient(new FooService(), new Http(), $this->getLogger());
    $this->renderText($client->call());
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
<?php

class ServiceClient {
    
    private $service;
    private $http;
    private $logger;
    
    public function __construct(Service $service, Http $http, sfLogger $logger) {
        $this->service = $service;
        $this->http = $http;
        $this->logger = $logger;
    }
    
    public function getService() {
        return $this->service;
    }

    public function setService(Service $service) {
        $this->service = $service;
    }
    
    public function call() {
        $this->logger->info("Calling service " . get_class($this->service) . " with url {$this->service->getUrl()}");
        $this->http->setUrl($this->service->getUrl());
        $this->http->setMethod($this->service->getMethod());
        $this->http->setBody($this->service->getBody());
        $this->http->send();
        return $this->http->getResponseBody();
    }
}

That’s better than before but will still have the symfony dependency in the constructor and a call to a method named info(). This could be solved by using an adapter, but who said we wanted any logging inside this class in the first place? Services, Service Clients and Http objects are all fairly cohesive, logging is not.

Implementing the Observer pattern

Shit just got real. We’ll now modify the Client so it implements SplSubject, the interface so kindly provided by PHP’s SPL. We’ll create a new Observer object that implements SplObserver and push the logging to it, then all that’s left is to modify the action code. So first up is the Client.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
<?php

class ServiceClient implements SplSubject {
    
    private $service;
    private $http;
    private $observers;
    
    public function __construct(Service $service, Http $http) {
        $this->service = $service;
        $this->http = $http;
    }
    
    public function getService() {
        return $this->service;
    }

    public function setService(Service $service) {
        $this->service = $service;
    }
    
    public function call() {
        $this->http->setUrl($this->service->getUrl());
        $this->http->setMethod($this->service->getMethod());
        $this->http->setBody($this->service->getBody());
        $this->http->send();
        $this->notify();
        return $this->http->getResponseBody();
    }
    
    public function attach(SplObserver $observer) {
        $this->observers[spl_object_hash($observer)] = $observer;
    }
    
    public function detach(SplObserver $observer) {
        unset($this->observers[spl_object_hash($observer)]);
    }
    
    private function notify() {
        foreach ($this->observers as $observer) {
            $observer->update($this);
        }
    }
}

You can see we are implementing the SplSubject interface where we have to declare 3 methods, attach(), detach() and notify(). As we are a subject we need to maintain an array of Observers, we do this by using a hash of the object as a key to the array which allows us to easily detach() it. notify() is our event where we loop over each Observer calling update() and passing them the current instance so they have full access to the context of when notify() was called. We call notify at the last moment so the observer has more information and available to it (thanks dol).

Next up the Observer.

1
2
3
4
5
6
7
8
<?php

class ClientSymfonyLoggerObserver implements SplObserver {
    
    private function update(SplSubject $subject) {
        sfContext::getInstance()->getLogger()->info("Calling service " . get_class($subject->getService()) . " with url {$subject->getService()->getUrl()}");
    }
}

Here you can see the object has one clear role – to use symfony’s logger and the Subject that we passed in (our Client) to log details about the Service that was called. We could easily create another Observer that uses Zend’s logger or create something that has a function other than logging. The point is we pushed the symfony dependency from inside the Client object into a separate, decoupled object.

And finally the action that brings it all together.

1
2
3
4
5
6
7
8
9
10
<?php

class WebServiceActions extends sfActions {
    
    public function executeWebServiceCall(sfWebRequest $request) {
        $client = new ServiceClient(new FooService(), new Http());
        $client->attach(new ClientSymfonyLoggerObserver());
        $this->renderText($client->call());
    }
}

You can see we construct our Client with 2 cohesive objects then attach an Observer directly after. We know that when notify() is called our logger will log details of the Service that was called. We’ve pushed the dependency out of the domain object and up into client code so that the domain object is no longer coupled to symfony, we could use it with Zend if we really wanted to (but probably don’t). As mentioned before, we could attach multiple Observers each with slightly different functionality. All we know is that each Observer wants to do something at the point that notify() was called. Nuff said.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Thu, 21 Apr 2011 05:25:00 -0700 Virgin Media Super Hub a.k.a. How Virgin Media effed my life http://blog.jmoz.co.uk/virgin-media-super-hub-aka-how-virgin-media-e http://blog.jmoz.co.uk/virgin-media-super-hub-aka-how-virgin-media-e

Ok, so at home we have Virgin Media Super Hub with the 50Mb package.  To cut a long story short, it's shit.  The following is a complaint from my housemate that he's sent to the top people at Virgin as we've had it to our wits' end with them.  I'm sticking it here for visibility and the hope that someone else can associate with us.

FYI. I'm not the typical complaining customer - i've been with Virgin since they were Blueyonder (and whatever before that) and the service was great, but the new service for the 50Mb Super Hub is the worst service i've ever had.  Ever.  We switched from Sky to Virgin on the promise it would be a great, fast service which it is not.  We never had any problems with Sky and i'm thinking of switching back - I can get a discount as well seeing as I work there :)

EDIT: There's an interesting thread regarding the SuperHub at this url, i've been getting a few referral hits off it http://cableforum.co.uk/board/12/33676964-superhub-nowhere-near-bad-people-say-page-32.html

 

Dear Mr Berkett,

 

Firstly, I hope this is the correct email address to contact yourself.

Secondly, I apologise for being one of those annoying customers who tries to bypass the usual complaints system and "go to the top" however after the 55 minute phone call I just had with your customer services team, I really don't know what else to do.

 

To give you a brief history of our account:

 

We signed up for 50mb broadband in September 2010.

The installation process was long & tedious.

The engineers didn’t have any routers with them, so we had to buy our own as getting one delivered was going to take so long.

We then had constant issues getting the virgin modem to connect correctly with the router we had bought, so we rung to complain and they mentioned we could have an all in one superhub which would rectify this issue. Happily we agreed and awaited arrival. Some weeks passed and no hub arrived. We called again to be told there was no record of an order for a superhub being made, and that the superhub wasn’t even available for 50mb customers yet. They apologised for giving incorrect information and told us that as soon the superhub was available we'd be sent one free of charge.

 

More weeks of bad connection, speed and intermittent issues passed. No superhub arrived. We called again to be told that we weren’t entitled to a free 50mb superhub so we would have to pay about £75 for it. Disgusted that we were being messed around again I spoke to a manager, explained all of our issues and if memory recalls correctly we eventually received a 50mb superhub for free.  

 

This is where the real problems began.

The engineer who was supposed to personally bring our superhuib and install it (according to the customer services team) had gone AWOL. So they would need to send another engineer out with another hub. When he arrived, he installed the hub and told us that we needed to fill in a registration page online. He left and we filled in the form. At the last page, the internet connection died and we never knew if the information we had just entered had been saved. We couldn’t seem to access the page again either.

The internet came back on but was riddled with issues. Our speed test results were around 2mb (nowhere near the 50mb we were paying for) the line test was giving us an F grade, with over 80% packet loss and a huge jitter score.

 

We called many times to complain, always receiving different excuses, usually related to "power levels being too high" and sometime too low!

We did a lot of online research and found an untold amount of virgin media customers had similar issues with the superhub.

 

Quoted from http://www.digitalspy.co.uk/forums/showthread.php?t=1449681

"If you check out the Virgin Media broadband forums you'll find the Superhub is plagued with problems and complaints from Superhub users. Virgin Media are fully aware that their Superhub has issues and have been promising a firmware upgrade for some time, but it has not happend yet. ETA for the Superhub firmware upgrade is reported to be May 2011"

 

As advised by many forums and eventually Virgin Media customer care, we switched off some of the router services such as IP Flood detection and the firewall. This drastically improved the line test score from an F to an A. We thought this had finally solved our problems.

 

Unfortunately, the service continued to drop out, usually when streaming media, or using FTP connections, it seemed to just not be able to handle any constant streams of data.

We called many, many times to complain  over the following few months, we've had 4 engineers out, some said they lowered the power levels, one added a filter to the line which would help. The last engineer told us there was something wrong with the line, but he couldn’t tell what. He advised us to find another broadband provider.

 

Completely disheartened by his comments, we called again to complain, we were promised a call back. No call back came.

Then began the billing issues.

The email address you had on file was incorrect, so we never received any bills.

One of the direct debits had failed because it was unusually high, so we were hit with late payment fees, and disconnected immediately.

No letters were sent, no communication whatsoever, assuming it was the usual technical issues we called the tech team, they advised us it was a billing issue so I was passed through to them.

They explained we had missed a payment (which i was at the time, oblivious to)

I made the payment and waited 24 hours for the broadband to come back on.

It did not.

 

I rung Virgin again having to use my mobile because you had cut our landline off now too. I complained again, and our internet was restored immediately.

About 4 days after this saga we received 3 letters (dated the day before) stating we had missed a payment, and that within 5 days we would be cut off. How useful is that?

I rang to complain again that sending these letters a week late is useless to us, they blamed the postal service, I stated that the letters were dated the day before the call, and they apologised.

 

Back to our usual poor connection, dropping out at many points throughout the day. Another 3 weeks passed and we were cut off again. Similar issues, no bill was received, late payment fee added, then removed. I rang up and paid, internet restored. At this time I explained how unhappy we were to be paying £80 a month and receiving such a terrible service from day one. A very helpful woman (for once) from customer services said she would arrange an engineer, I said we'd already had 4 too many and she seemed shocked that it still wasn’t resolved. She did a line test and agreed there was a problem. She said she would escalate it to the highest level of complaints and investigation to get this sorted once and for all, then look at compensation.

 

A week passed and no call back was received, then we were cut off AGAIN. This is today, I call and am told that £10 is still outstanding, I ask what it's for and they advise that when the last customer service representative took off the last late payment fee, they wrongly charged us £10 less, when in fact the £10 credit was added to next month's bill, so £10 was left on the account unpaid.

We had received no letters or warning as usual, just cut off out of the blue. (I'm fairly sure this isn’t policy at all, you're supposed to warn customers before cutting them off)

Angered by yet another out of the blue disconnection, I asked to be put through to cancellation to close the account. I was told I couldn’t do that until I paid £10.

I was also advised that there would be a £180 cancellation fee. I argued that we were expecting a call regarding the poor service and compensation, I was then put through to a "manager", Carl.

This guy was the final straw.  Comparing the service to a 'Rolls Royce' because it's so good, he argued that we had clearly had a great service due to our high download usage. I advised him that's because I work from home a lot, and there are 5 of us living here. He then suggested that was the problem, that too many machines were draining the bandwidth. I advised that was a silly comment, and something that we had clearly addressed months back, long before we had 4 engineers, and escalated complaints etc.

 

He then suggested that sometimes kitchen appliances can cause problems. I asked if he was in the technical team, he admitted he was not. So we ended that conversation.

He then quite rudely, stated that the only way we would have our connection restored was if we paid £10. I said out of principal alone, I'm not paying any more money. He said "well then you won't have our great service" he kept dropping comments like this at me, and that we download too much anyway. I advised its an unlimited service, so we're well within our rights to download a lot of content. He agreed and stated that no other provider offers that so it's not like we can go elsewhere. I advised that BT offer unlimited 40Mb broadband. He argued that they only provide 24 megabyte speeds, and virgin offers 50megabytes. I advised Virgin do NOT offer 50Megabytes. They offer 50Megabits. I asked him to learn the difference between bits  and bytes if he's going to lecture me on what I am and am not entitled to spend my money on.

He then returned to his usual script of "You have to pay £10" I asked to speak to his manager, he said that would take up to 24 hours. I asked to speak to one immediately,. He said no.

I asked to speak to one of his colleagues now, he said no. Eventually, I paid the £10. I asked him to log the complaint on the system with all of the other complaints. That phon ecall took 55 minutes. And now I'm emailing you hoping that something will happen.

 

We're not asking for much, just one or possibly if you can stretch to it, 2 of the following:

 

1) Some form of cash compensation

2) You stop cutting us off

3) you fix our broadband line issues

 

One would hope, that those 3 are the basics that you could expect from a £45 a month broadband service.

 

P.S - Yesterday, our TV remote packed up too. We replaced the batteries but it's still dead. If I didn't laugh about this, I'd cry.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Wed, 13 Apr 2011 11:28:00 -0700 SCRUM planning poker iPhone app - Agile Poker http://blog.jmoz.co.uk/scrum-poker-planning-iphone-app http://blog.jmoz.co.uk/scrum-poker-planning-iphone-app

If you practice SCRUM at your workplace you should be familiar with planning poker - where the team sit down and try and come up with estimates for a user story showing numbered cards usually using the fibonacci sequence.

 

Seeing as 90% of the team have iPhones why not try and get rid of the boring cards and replace them with an iPhone app - Agile Poker. It does the job. Pick numbers easily, bright colours so they stand out and there's a few customisation options. Plus, you can look on Twitter or Facebook and pretend you're picking your estimates.

Screenshots and a demo follow!

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Wed, 13 Apr 2011 08:15:33 -0700 Shoreditch PHP? http://blog.jmoz.co.uk/shoreditch-php http://blog.jmoz.co.uk/shoreditch-php

If you've got a decent PHP contract in the Shoreditch area, hook a brother up.  I want to ride my fixie bike to work.

20100129-old-street-map

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Fri, 11 Mar 2011 03:04:55 -0800 Firewatir and sudo gem update --system http://blog.jmoz.co.uk/sudo-gem-update-system http://blog.jmoz.co.uk/sudo-gem-update-system

Trying to install Ruby, Cucumber and Firewatir on Ubuntu Maverick leads to a problem i'm sure i've seen before.  I've installed Ruby/Rails numerous times on Linux boxes and it never ever works first time.  For the amount of time Ruby has been around you'd think they'd get it to work on a Linux/Debian install easily (like PHP/MySQL/Apache), but no, always a frigging problem.

Anyway, i've used apt-get to install ruby and rubygems.  From that you can then install the cucumber gem ok but the firewatir gem breaks:

 $ sudo gem install firewatir ERROR:  Error installing firewatir:         hoe requires RubyGems version >= 1.4. Try 'gem update --system' to update RubyGems itself.  

If you then try sudo gem update --system you'll get:

$ sudo gem update --system ERROR:  While executing gem ... (RuntimeError)     gem update --system is disabled on Debian, because it will overwrite the content of the rubygems Debian package, and might break your Debian system in subtle ways. The Debian-supported way to update rubygems is through apt-get, using Debian official repositories. If you really know what you are doing, you can still update rubygems by setting the REALLY_GEM_UPDATE_SYSTEM environment variable, but please remember that this is completely unsupported by Debian.  

The reason it won't work is that the packaged version that's in the Maverick repositories is 1.3x.  You need to remove the package using apt-get and then use the latest source instead:

 $ sudo apt-get remove rubygems

Then download rubygems from their website at http://rubygems.org/pages/download.  Once you've extracted it and gone into the directory run:

 $ sudo ruby Downloads/rubygems-1.6.1/setup.rb  RubyGems 1.6.1 installed   

You'll have to re-install the Cucumber gem but now the Firewatir gem should work:

 $ sudo gem install firewatir Fetching: xml-simple-1.0.14.gem (100%) Fetching: rake-0.8.7.gem (100%) Fetching: hoe-2.9.1.gem (100%) Fetching: s4t-utils-1.0.4.gem (100%) Fetching: user-choices-1.1.6.1.gem (100%) Fetching: commonwatir-1.8.0.gem (100%) Fetching: firewatir-1.8.0.gem (100%) Successfully installed xml-simple-1.0.14 Successfully installed rake-0.8.7 Successfully installed hoe-2.9.1 Successfully installed s4t-utils-1.0.4 Successfully installed user-choices-1.1.6.1 Successfully installed commonwatir-1.8.0 Successfully installed firewatir-1.8.0 7 gems installed  

More info here:

http://stackoverflow.com/questions/3643870/gem-update-system-is-disabled-on-debian-error

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Fri, 07 Jan 2011 08:51:00 -0800 Double Netbeans icon on Docky http://blog.jmoz.co.uk/post/2638218912 http://blog.jmoz.co.uk/post/2638218912

For the past couple of weeks i’ve been putting up with a bug where my Docky shows two instances of Netbeans.  The icon on the dock acts as a launcher which creates another Netbeans icon instead of using just the single icon and lighting up.  The problem is due to Netbeans being a Java app and is also down to the way Docky handles window matching.

I did a bit of googling and found the cure at http://www.issathen.co.uk/?p=14.

To get Docky to work nicely with Netbeans 6.9.1, create a file, netbeans-6.9.1.desktop, and stick the following contents inside it:

#!/usr/bin/env xdg-open [Desktop Entry] Encoding=UTF-8 Name=NetBeans IDE 6.9.1 Comment=The Smart Way to Code Exec=/bin/sh "/home/foo/netbeans-6.9.1/bin/netbeans" Icon=/home/foo/netbeans-6.9.1/nb/netbeans.png Categories=Application;Development;Java;IDE Version=1.0 Type=Application Terminal=0 StartupWMClass=java-lang-Thread

Obviously adjust the paths so they match up with your Netbeans location.  Alternatively, copy the Netbeans icon from your Ubuntu Applications -> Programming menu bar to your desktop and edit it.

Either way, once you’ve got the file with the crucial StartupWMClass=java-lang-Thread appended to the bottom, drag it onto your Docky and it should work correctly!

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Mon, 27 Dec 2010 09:33:00 -0800 Mixcloud downloader - clouddownload.co.uk http://blog.jmoz.co.uk/post/2484380214 http://blog.jmoz.co.uk/post/2484380214

I use Mixcloud all the time.  My iPhone is loaded with mixes from the site.  A common complaint heard from users is that they don’t provide you with download links, so I wrote a little app that parses their JSON and gets you a list of the mp3 download links.  I’ve also added some features like a list of recently downloaded cloudcasts and a formatted tracklist that you can paste in your iTunes lyrics field.  I’ve uploaded it to my VPS and pointed a domain to it.

So if you want to download Mixcloud mp3s, check out clouddownload.co.uk.

I’m working on adding support for Soundcloud as well, so look out!

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Wed, 08 Dec 2010 02:44:27 -0800 Skype plugin for Pidgin on Ubuntu Lucid crashes! http://blog.jmoz.co.uk/post/2142263537 http://blog.jmoz.co.uk/post/2142263537

Aaaarrgghhh!  Someone please fix this!

After installing the pidgin-skype package, then when trying to add a Skype account in Pidgin it crashes.  The output of pidgin -d gives:

(10:06:24) account: Connecting to account someaccount.
(10:06:24) connection: Connecting. gc = 0x2904170
(10:06:24) GLib: GError set over the top of a previous GError or uninitialized memory.
This indicates a bug in someone's code. You must ensure an error is NULL before it's set.
The overwriting error message was: Failed to execute child process "skype" (No such file or directory)

I just want to be able to use Skype and MSN alongside each other in Pidgin.  Please make it work.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Mon, 10 May 2010 14:10:34 -0700 Untitled http://blog.jmoz.co.uk/post/587552792 http://blog.jmoz.co.uk/post/587552792

Tumblr_l2801mtqjh1qbpafjo1_1280

Holy shit!  Android Hello World app… check!  Next step, Android marketplace domination…

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris
Sat, 24 Apr 2010 09:32:59 -0700 Untitled http://blog.jmoz.co.uk/post/545715573 http://blog.jmoz.co.uk/post/545715573

Tumblr_l1e0izgcd61qbpafjo1_1280

This is one of the best Mario images i’ve ever seen.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1004916/LoPan.jpg http://posterous.com/users/he6CoYULbcxJM James Morris jmoz James Morris