Category Archives: zend framework

How do I send emails in my applications

Today I decided to share some of my ideas according to email sending. I will use yii app as an example but it’s absolutely not required. It’s just an aproach that you can use with any other framework. Right now yii doesn’t have default mail component. But I won’t tell you what mail extension or lib should you use. I want to tell you about how should look like email send in your app.

Well I really believe that you already don’t send emails like this 🙂

$message = " Pellentesque convallis tempor tortor. Nullam nec purus.";
mail('somebody@example.com', 'Nonsensical Latin', $message);

But you do it like this

     $message = \Swift_Message::newInstance()
        ->setSubject('Hello Email')
        ->setFrom('send@example.com')
        ->setTo('recipient@example.com')
        ->setBody(
            $this->renderView(
                'HelloBundle:Hello:email.txt',
                array('name' => $name)
            )
        )
    ;
    $this->get('mailer')->send($message);

It could be Swift or Zend Mailer or other good lib.

What are advantages in this approach

  • First of all you don’t create a wheel but use good and checked tools
  • It doesn’t render email body inside your code. It renders it from some view. Very good, take five

But what are disadvantages in this approach

  • it’s not DRY. In real app you will send dozens of emails. And following this approach you’ll have dozens places with the same piece code. 
  • it’s doesn’t scalable at all if tomorrow you want to log sent emails or want to render it in html, or add bcc etc, you have to to add additional logic in dozen of places. 
  • it’s quite difficult to update email template. If we have some managers, moderators, admin that sometimes want to improve some text in email they can’t do that without asking developer.

So what is my proposal?

Email send have to look like this

$emailReset = new \MyApp\email\ResetPassword(array(
    'user' => $userModel,
    'hash' => $hash,
    /* other dynamic data for email */
));
$emailReset->send();

And that’s it! You don’t need another code. You create an obejct of required email, in my example it’s reset password email. I pass user model, so then we can get email, username, etc and I pass specific data for that email. And… I send it.

How we can achieve that?
We have various emails in our app – sign in, reset password, sale emails, promo emails, and many many others. For each email we know subject and body.  The idea is to keep email templates in DB

email

Nothing fancy at that point. Then I add parent class for all my emails

<?php

namespace MyApp\email;

abstract class Base
{

    /**
     *
     * @var TransportInterface
     */
    private static $_transport;

    /**
     *
     * @var \MyApp\model\EmailTemplate
     */
    private $_emailTemplate;
    protected $_emailCode;
    private $_addToEmail;
    private $_addToUsername;
    private $_data;

    /**
     * Recipient
     * @var \MyApp\model\User
     */
    private $_user;

    public function __construct($data)
    {
        $data = $this->_initData($data);

        if (!empty($data['user']) && ($data['user'] instanceof \MyApp\model\User)) {
            $this->_user = $data['user'];
            $data['email'] = $this->_user->email;
            $data['username'] = $this->_user->getUsername();
        } else {
            if (empty($data['email'])) {
                throw new \Exception("Email is not defined");
            }
        }
        $this->_addToEmail = $data['email'];
        if (!empty($data['username'])) {
            $this->_addToUsername = $data['username'];
        }
        $this->_data = $data;
    }

    protected function _initData($data)
    {
        return $data;
    }

    /**
     * Compile and send email
     */
    public function send()
    {
        $email = $this->_emailTemplate();

        $email->compileEmail($this->_data);

        $defaultEmailParams = \Yii::app()->params['email'];
        $mail = new \Zend\Mail\Message();
        $mail->setBody($email->body);
        $mail->setSubject($email->subject);
        $mail->addTo($this->_addToEmail, $this->_addToUsername);
        $mail->setFrom($defaultEmailParams['defaultFromEmail'], $defaultEmailParams['defaultFromName']
        );

        //$mail->addHeader('List-Unsubscribe', '<' . 'link' . '>');
        $this->getTransport()->send($mail);

        return true;
    }

    /**
     * Return email transport
     * @return \Zend\Mail\Transport\TransportInterface
     */
    public static function getTransport()
    {
        if (!self::$_transport) {
            if (\Yii::app()->params['email']['provider'] == 'sendmail') {
                self::$_transport = new \Zend\Mail\Transport\Sendmail();
            } elseif (\Yii::app()->params['email']['provider'] == 'file') {
                $fileOptions = new \Zend\Mail\Transport\FileOptions();
                $fileOptions->setPath(\Yii::app()->basePath . '/runtime/cache/email/' . APPLICATION_ENV);
                self::$_transport = new \Zend\Mail\Transport\File($fileOptions);
            }
        }
        return self::$_transport;
    }

    /**
     * Return email object
     * @return \MyApp\model\EmailTemplate
     * @throws Exception
     */
    protected function _emailTemplate()
    {
        if (!$this->_emailCode) {
            throw new Exception("Template name is not set");
        }
        if (!$this->_emailTemplate) {
            $this->_emailTemplate = \MyApp\model\EmailTemplate::getByCode($this->_emailCode);
        }

        return $this->_emailTemplate;
    }

}

It’s just an example. So your class be different could more functionality that I described above. I show just an approach. I used ZendMail component it’s so easy now if you user composer (  “zendframework/zend-mail”: “*” ) but you wanna use what ever you want. I believe everything is quite straight forward here.

Only the one note about $email->compileEmail($this->_data) I liked to use Twig to render emails. So compileEmail in fact just render twig template that we have in db and  copy in file. But it’s another topic. If anyone would be interested I can describe that in more detail.

Finally the example of specific email is following

<?php

namespace MyApp\email;

class ResetPassword extends \MyApp\email\Base
{

    protected $_emailCode = 'reset_password';

    protected function _initData($data)
    {
        $data['link'] = \Yii::app()->createAbsoluteUrl('urltoreset', array(
            'email' => $data['user']['email'],
            'hash' => $data['hash']
        ));

        return $data;
    }

}

That’s it. Hope it would be helpful for someone 🙂

Converting Zend config from ini to yaml. Non-obvious problems

I first knew about yaml from rails. And I really liked me. But unfortunatelly there wasn’t built in support of yaml in Zend Framework. However I started use yaml in my ZF apps. I added simple component that uses Symfony parser.

<?php

include_once 'SymfonyComponents/YAML/sfYaml.php';

/**
 * Yaml decoder from Symfony
 * @author: radzserg
 * @date: 05.10.11
 */
class App_Yaml
{

    /**
     * Load YAML
     * @static
     * @param $input file|yaml formatted text
     * @return array
     */
    public static function load($input)
    {
        $sfYaml = new sfYaml();
        return $sfYaml->load($input);
    }

    /**
     * Dump array into YAML formatted text
     * @static
     * @param $array
     * @return string
     */
    public static function dump($array)
    {
        $sfYaml = new sfYaml();
        return $sfYaml->dump($array);
    }

}

And finally in ZF 1.11.12 (I could mistaken here) Zend_Config_Writer_Yaml was added. So I’ve decided to convert application.ini to application.yml. Cause config in one of my project became bigger and bigger. I found ready solution in some blog and decided to use it.

$inputfile = APPLICATION_PATH . '/configs/application.ini';
$outputfile = APPLICATION_PATH. '/configs/application.yml';

$config = new Zend_Config_Ini($inputfile, null, array('allowModifications' => false, 'skipExtends'=> true));
$writer = new Zend_Config_Writer_Yaml();
$writer->write($outputfile, $config, true, true);
echo file_get_contents($outputfile);

Very simple and clear. I’ve scan received file. Looks good. All tests are passed. And I do the same on live server.

After few days we started getting errors about General error: 1205 Lock wait timeout exceeded; for some tables. I was checking mysql by SHOW PROCESSLIST and can’t find any blocking/competing query. I asked our admins to help but didn’t get any valuable info from them. The problem deal only with InnoDB tables, 90% of tables in our db are MyISAM. We were trying to check “show engine innodb status” but also found only not started transactions. Finally I tried to restart gearman workers (they are daemons and work for for a long time in Mysql) maybe this could help…
And I could execute the query. I tried to add force commit after every task execution. That helps. But the reason was still uncleared for me. Next day one of our customer couldn’t update some info. He did the query but nothing happened. I thought that admins could switch off AUTOCOMMIT for InnoDB in Mysql. Tried to do show variables like “%autocommit%”; from shell and got “ON”. Tried to do the same from php side and !!! got “OFF” how this could happen??? Everything worked ok untill this moment.

I looked once again into config and what did I find

resources.multidb.dbname.adapter = “pdo_mysql”
resources.multidb.dbname.host = “localhost”
resources.multidb.dbname.username = “user”
resources.multidb.dbname.password = “pass”
resources.multidb.dbname.dbname = “dbname”
resources.multidb.dbname.driver_options.1002 = “SET NAMES utf8;”

was connverted to

resources: 
  multidb: 
    dns: 
      adapter: pdo_mysql
      host: localhost
      username: user
      password: pass
      dbname: dbname
      driver_options: 
        - SET NAMES utf8;

Look at SET NAMES utf8 and see that 1002 key is missing and now it’s equal to 0. And if you look into Zend_Db you’ll find
// PDO constant values discovered by this script result:
const ATTR_AUTOCOMMIT = 0;

And finally the root of the problem Zend_Config_Writer_Yaml

  /**
     * Service function for encoding YAML
     *
     * @param int $indent Current indent level
     * @param array $data Data to encode
     * @return string
     */
    protected static function _encodeYaml($indent, $data)
    {
        reset($data);
        $result = "";
        $numeric = is_numeric(key($data));                // look here

        foreach($data as $key => $value) {
            if(is_array($value)) {
                $encoded = "\n".self::_encodeYaml($indent+1, $value);
            } else {
                $encoded = (string)$value."\n";
            }
            $result .= str_repeat("  ", $indent).($numeric?"- ":"$key: ").$encoded;    // and look here
        }
        return $result;
    }

The most interesting that Symfony will convert it correct. Try to add $writer->setYamlEncoder(array(‘App_Yaml’, ‘dump’));

and it will work

Be careful when it deals with configs.

Prevent General error: 2006 MySQL server has gone away in Zend Framework

As introduction

I started to get General error: 2006 MySQL server has gone away error in one of my project. I’ve read http://dev.mysql.com/doc/refman/5.0/en/gone-away.html but still was not sure why does it happen. Until I stared use php Gearman workers. That was daemon processes they work in memory all the time. I had 5 workers and all of them have such error. I realized that workers sleep between tasks and do not send any command to mysql and mysql close connection. Such problem can happen with cron script that works for a long time but has delays between queries(do some other tasks).

Solution

One of the things for which I like Zend Framework is really easy extensibility. We can just override default Adapter and add some functionality. So the idea is simple if we get 2006 error during query we will try to reconnect. If it didnt’t help we will throw original exception.

<?php

/**
 * Extended Zend_Db_Adapter_Pdo_Mysql - try to reconnect if mysq gone away
 * User: radzserg
 * Date: 4/19/12
 */
class App_Db_Adapter_Pdo_Mysql extends Zend_Db_Adapter_Pdo_Mysql
{

    const MAX_RECONNECT_COUNT = 1;

    public function query($sql, $bind = array())
    {
        try {
            return parent::query($sql, $bind);
        } catch (Zend_Db_Statement_Exception $e) {
            $message = $e->getMessage();
            if ($message == 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away') {
                $this->_reconnect();
                return parent::query($sql, $bind);
            } else {
                throw $e;
            }
        }
    }

    private function _reconnect()
    {
        $this->_connection = null;
        for ($i = 1; $i <= self::MAX_RECONNECT_COUNT; $i++) {
            sleep(1);
            try {
                $this->_connect();
            } catch (Zend_Db_Adapter_Exception $e) {
                if ($i == self::MAX_RECONNECT_COUNT) {
                    throw $e;
                }
            }
            if ($this->_connection) {
                return ;
            }
        }
    }
}

All we have to do now is just add appropriate instruction to config
resources.multidb.default.adapterNamespace = “App_Db_Adapter”

That’s it feedback is welcome.

Generate phpunit fixtures

Last time I had a lot of work and that’s why can’t write something really interesting. So today I decided to write about another one zf command (read how do I do it here Run Zend Framework Command in Yii style) that helps me to build fixtures for PHPUnit.
Imagine you need to quickly create a fixture for some tables with many fields. Writing tests should not be time-consuming. So you do something like this

php script_runner.php createFixtureFromDb --table=user --print=1 --limit=1 --verbose

<?xml version="1.0" encoding="UTF-8"?>
<dataset><tags id="1" login="login" email="email@email.com" a lot of other fields etc.. />
</dataset>

Shortly how do I do that –

<?php

/**
 *
 * Creates fixture for testing from specified DB table
 * @author radzserg
 *
 */
class App_Script_Command_CreateFixtureFromDb extends App_Script_Command_Abstract
{

    protected $_availableParams = array(
        'table*' => 'Name of the table to import data',
        'file' => 'Path generated xml file. (Either file or print shoud be specified)',
        'print' => 'Output to stdout',
        'limit' => 'limit of rows',
        'rand' => 'if set will add order by RAND()',
    );

    public function execute($params = array())
    {
        if (empty($params['table'])) {
            throw new Exception("Specify source table");
        }
        if (empty($params['file']) && !isset($params['print'])) {
            throw new App_Exception_System("Specify file path for xml");
        }
        $limit = isset($params['limit']) ? (int)$params['limit'] : null;
        $cond = isset($params['cond']) ? $params['cond'] : null;
        $rand = isset($params['rand']) ? true : false;

        $table = $params['table'];
        $filePath = isset($params['file']) ? $params['file'] : null;

        /* @var $adapter Zend_Db_Adapter_Abstract */
        $adapter = Zend_Db_Table::getDefaultAdapter();
        $select = $adapter->select()->from($table);

        if ($rand) {
            $select->order(new Zend_Db_Expr('RAND()'));
        }
        $select->limit($limit);
        if ($cond) {
            $select->where($cond);
        }

        $rows = $adapter->fetchAll($select);

        $doc = new DomDocument('1.0', 'UTF-8');
        $root = $doc->createElement('dataset');
        $root = $doc->appendChild($root);

        foreach ($rows as $row) {
            $xmlRow = $doc->createElement($table);
            foreach ($row as $col => $value) {
                $xmlRow->setAttribute($col, $value);
            }

            $root->appendChild($xmlRow);
        }

        $xml = $doc->saveXML();

        if (!isset($params['print'])) {
            file_put_contents($filePath, $xml);
        } else {
            $this->verbose($xml, 'info');
        }
    }

}

As you can see from $_availableParams help variable we have to set required table param. Other params are optional. You can specify –limit for rows, –file – file path param if you want to save result to file, –print if you want to output result to console, and finally you can specify –rand variable if you want to randomize result.

That’s it, hope it was helpfull.

Zend_Test_PHPUnit_DatabaseTestCase with multidb

Today I’ll tell you how you can run tests for multiple databases with PHPUnit_DatabaseTestCase at the same time.There are 2 ways. First is very simple you can use in your fixtures something like this

It will work if only you have enough rights it’s good for development/local db. But as a rule on even dev servers it won’t work.

Second approach is to extend PHPUnitsetUp
I’ve changed some names you can use more clear names in your projects.

abstract class App_Test_PHPUnit_DatabaseTestCase extends Zend_Test_PHPUnit_DatabaseTestCase
{
 
    protected $_connectionMock;
    protected $_secondDbConnectionMock;
 
    protected $backupGlobalsBlacklist = array('application');  // btw this hack will speed up your tests
 
    /**
     * @return Zend_Test_PHPUnit_Db_Connection
     */
    protected function getConnection()
    {
        if ($this->_connectionMock == null) {
            $multiDb = Zend_Registry::get('multidb');
 
            $connection = $multiDb->getDb();
 
            $this->_connectionMock = $this->createZendDbConnection(
                $connection, ''
            );
 
            Zend_Db_Table_Abstract::setDefaultAdapter($connection);
        }
        return $this->_connectionMock;
    }
 
    protected function getSecondDbConnection()
    {
        if ($this->_dnsConnectionMock == null) {
            $multiDb = Zend_Registry::get('multidb');
 
            $connection = $multiDb->getDb('second_db');
 
            $this->_secondDbConnectionMock = $this->createZendDbConnection(
                $connection, ''
            );
        }
 
        return $this->_dnsConnectionMock;
    }
 
    protected function setUp()
    {
        parent::setUp();
 
        $this->databaseTester = NULL;
 
        $this->getDatabaseTester()->setSetUpOperation($this->getSetUpOperation());
        $this->getDatabaseTester()->setDataSet($this->getDataSet());
        $this->getDatabaseTester()->onSetUp();
 
        $secondDataSet = $this->getDataSetForSecondDb();
        if ($dnsDataSet) {
            // create data set for second db
            $secondDataTester = new PHPUnit_Extensions_Database_DefaultTester($this->getSecondDbConnection());
            $secondDataTester->setSetUpOperation($this->getSetUpOperation());
            $secondDataTester->setDataSet($secondDataSet);
            $secondDataTester->onSetUp();
        }
 
    }
 
    protected function getDataSetForSecondDb()
    {
        return null;
    }
 
}

as you see we just check does method getDataSetForSecondDb return data. You have to override it in child classes. If we get dataSet we will set up operations for second Db.

Everything is quite simple but in the fullness of time it made me to  look inside PHPUnit_* classes. So I hope I can save you time with that stuff.

Run Zend cli scripts (yii style)

yii has a nice yiic tool – cli for *nix so you can run scripts in this way

./yiic mynicescript –param1=2 –param2=2

this is really cool feature is essentially just a bootstrap for for running cli yii scripts(commands).

It’s weird that Zend Framework(ZF) doesn’t have something similar. I propose you my variant for ZF.

There are 2 classes (btw some features are similar with Extended Yii CConsoleCommand:) described earlier.

<?php

/**
 *
 * Implements function to work from CLI
 * @author radzserg
 *
 */
abstract class App_Script_Abstract
{
    const VERBOSE_ERROR = 'error';
    const VERBOSE_INFO = 'info';

    protected $_verbose;
    
    /**
     *
     * Return class name
     * @return string
     */
    public function getName()
    {
        return get_class($this);
    }
    
    /**
     * Get params from cli
     * myscript.php --param1=value --param2=value
     * @return array
     */
    public function getCliParams()
    {
        $params = isset($_SERVER['argv']) ? $_SERVER['argv'] : array();
        $resultParams = array();
        foreach ($params as $paramPair) {
            if (strpos($paramPair, '--') !== false && strpos($paramPair, '--') == 0) {
                $count = 1;
                $paramPair = str_replace('--', '', $paramPair, $count);
                $paramPair = explode('=', $paramPair, 2);
                $key = isset($paramPair[0]) ? $paramPair[0] : null;
                $value = isset($paramPair[1]) ? $paramPair[1] : '';
                if ($key) {
                    $resultParams[$key] = $value;
                }
            }
        }
        return $resultParams;
    }

    /**
     * Verbose info
     * @param $message
     * @param null $type
     */
    public function verbose($message, $type = null)
    {
        if ($this->_verbose === NULL) {
            $cliParams = $this->getCliParams();
            $this->_verbose = isset($cliParams['verbose']) ? true : false;
        }
        if ($this->_verbose) {
            if ($type == self::VERBOSE_ERROR) {
                // message in red
                echo date('H:i:s ') . "\033[31;1m" . $message . "\033[0m\n";
            } elseif ($type == self::VERBOSE_INFO) {
                // message in green
                echo date('H:i:s ') . "\033[32;1m" . $message . "\033[0m\n";
            } else {
                echo date('H:i:s ') . $message . "\n";    
            }
        }
    }
}

<?php

class App_Script_ScriptRunner extends App_Script_Abstract
{

    /**
     *
     * Run script
     */
    public function run()
    {
        $script = $this->_getScriptObject();
        $params = $this->getCliParams();
        if (isset($params['help'])) {
            $script->help();
        } else {
            $script->execute($params);    
        }
    }
    
    
    /**
     *
     * Return script nam that have to be run
     * Mandatory format
     *
     * cron_runner.php scriptName [--param1=value --param2=value]
     * @return string
     */
    public function getScriptName()
    {
        return isset($_SERVER['argv'][1]) ? $_SERVER['argv'][1] : NULL;
    }

    
    /**
     *
     * Return script object instance of App_Exception_System
     * @return App_Script_Abstract
     * @throws App_Exception_System
     */
	protected function _getScriptObject()
	{
	    $scriptName = $this->getScriptName();
        if (!$scriptName) {
            throw new App_Exception_System("Cron name is not defined. Server argv " . print_r($_SERVER['argv'], true));
        }
        
        $scriptName = "App_Script_Command_" . ucfirst($scriptName);
        $script = new $scriptName;
        
        return $script;
	}
    
}

Second class App_Script_ScriptRunner is a descendant class and you can extend it as you want. As you see in my example I extended _getScriptObject my scrips are in /App/Script/Command perhaps just /scripts is better place. For example  I use another one App_Script_CronRunner to run cron scripts that additionally adds some stat about script execution.

And finally you have to add entry script for CLI. I put it in /scripts folder.

/**
 * Run CLI scripts
 *
 * php cron_runner.php myCommandName --param1=value --param2=value
 * command should be located in App/Script/Command/
 */

$envention = file_get_contents(dirname(__FILE__) . '/cli_envention');
if (!$envention) {
    throw new Exception("CLI envention is not defined");
}
define('APPLICATION_ENV', $envention);
define('ROOT_PATH', realpath(dirname(__FILE__) . '/..'));

define('APPLICATION_PATH', ROOT_PATH . '/application');

set_include_path(implode(PATH_SEPARATOR, array(
    realpath(ROOT_PATH . '/library'),
    get_include_path(),
)));

if ('development' == APPLICATION_ENV) {
    error_reporting(E_ALL | E_NOTICE);
    ini_set('display_errors', 1);
}

require_once 'App/Application/Console.php';
// Create application, bootstrap, and run
$application = new App_Application_Console(
    APPLICATION_ENV,
    APPLICATION_PATH . '/configs/application.ini'
);
$application->bootstrap();

$scriptRunner = new App_Script_ScriptRunner();
$scriptRunner->run();

As you see it’s very similar to index.php The last hack that I use here is App_Application_Console it’s full copy of Zend_Application but uses his own Bootstrap class.

That’s it now you can run your scripts in this way.

cd /scripts
php script_runner.php haveFun –param1=123 –verbose

I forgot to remove debug code and updated last changes to production

Sometimes you need to debug some code and you do something like this

if ($service->canBeUpdated()) {
    // $service->canBeUpdated returns false 
    // and you need to debug this place 
}

// and you do simple (very simple) stub
if (true || $service->canBeUpdated()) {
}

😀 ooh everything works and you can easily debug this place. You’ve finished with that task and did some other tasks and finally updated everything to live.

Oh crap.. I forgot to remove debug code and updated last changes to production (or some customer complains that smth is wrong and this is even worst)

So I use this simple component in order to prevent such situations. Because to be honest I faced with such problems not one time.

<?php
 
/**
 *
 * User: radzserg
 * Date: 3/6/12
 */
  
class App_Debug
{
 
    /**
     * @static
     * @param $val
     * @return bool
     */
    public static function alwaysTrueForDebug($val)
    {
        if (APPLICATION_ENV == 'development') {
            return true;
        } else {
            return $val;
        }
    }
 
    public static function setVarForDebug(&$var, $value)
    {
        if (APPLICATION_ENV == 'development') {
            $var = $value;
        }
    }
 
}
// Now I can do it in this way 
if (App_Debug::alwaysTrueForDebug($service->canBeUpdated)) {
// or
App_Debug::setVarForDebug($user['state'], 'active')
 
// first expression will be true and $user['state'] will be set 
// if only APPLICATION_ENV == 'development'