Caching your views to improve the performance of your PrestaShop Modules

In this post I will talk about a very important topic in the development of PrestaShop modules: how to cache your template files to improve the performance of the modules. The relevance of this topic is even greater when we talk about Front-Office modules, because a bad written module may have serious impact in the loading time of the shop. The good news is that is very easy to use the PrestaShop own caching system.

As you may already know, all the HTML of the shop is (or should be) generated through some template language (usually smarty). As they are template languages, they must be compiled for the HTML to be generated. If no cache is used, every time a page is accessed, several template files have to be compiled, even if they didn’t change since the last compilation. This is a waste of server resources, and also makes the page to take more time to be load than it would be necessary. Thins get even worse if several database queries are performed to get the data used in the template.

As has already been said, the caching system of PrestaShop is very easy to be used, and you just have to include a few lines of code to your module. Let’s suppose that your module insert some content in the  Front-Office through the displayShoppingCartFooter hook. So, your code will be something like

When this function is executed, if there is a cached version of the template ‘mytemplate.tpl’, PrestaShop will automatically use the cached version, instead of recompile it. If your template didn’t depends of the $data variable, we wouldn’t need do to nothing more. However, as the value of the shopping cart changes, the HTML generated by this template may change as well. This makes necessary the use of some way to distinguish between different cached versions of the same template file.

These different versions may be produced using the third argument of the method Module::display(), named $cache_id, as the following code:

Now, we have different cached versions of the template file, and each version is indexed by the shopping cart value. It is a good practice to use the method Module::getCacheId(), which returns a string with the following information:

  • Whether the shop was accessed using secure connection or not;
  • ID of the shop (if using multishop);
  • ID of client group which the user who accessed the shop belongs to;
  • Language in use by the client;
  • Currency in use by the client;
  • Country in use by the client.

Then, a very good cache_id may be obtained like the following code.

Note that the variable $data is not used when the template is already in cache, because the template will not be compiled. So, if the template is in cache, the method MyModel::getSomeDbData() shouldn’t be called. Then, we can save some CPU resources with the following code.

Finally, the cache invalidation may be done with the method Module::_clearCache(). If we want to clear every cached version of our template file whenever any object of the class MyModel is updated, we may use the hook actionObjectMyModelUpdateAfter. This hook is called by PrestaShop whenever the method update() is executed with success in a object of the class MyModel (which must extends the PrestaShop class ObjectModel).

The method Module::_clearCache() also accepts the argument $cache_id, allowing us to clear the cache of a single version of our template file.

Developing a simple module with CRUD for PrestaShop

In this article, we will develop a module to be used in the administrative panel of the shop. This module implements the four basic CRUD operations, taking advantage of a lot of features that is already present in the PrestaShop’s core. Our forms and grids will all be generated automatically by the PrestaShop, in such a way that they will have the same layout as the pages of the shop. Also, it should make the module compatible with newer versions of the PrestaShop.

The following topics will be detailed:

  1. Automatic creation of database tables;
  2. Automatic reparation of the database in the module reinstallation;
  3. Listing data in a grid using only PrestaShop core features;
  4. Creating forms to create/update data using only PrestaShop core features;
  5. Deleting data from the database using individual and bulk actions.
  6. Utilization of multilanguage fields.

The items 1 and 2 will use the CustomObjectModel class. I developed this class to add some features to the PrestaShop’s ObjectModel class. So, our Model (we will have only one) must be defined in the same way that the PrestaShop’s ones are defined. I will show how to to this definition in this article. Once the model is defined, the 3., 4. and 5. items will demand little effort; the hard work is already implemented in the core of PrestaShop.

The full source of the module is available in this GitHub repository. You just have to extract it to /modules/crud folder. I strongly recommend that you read this article using the module in your own PrestaShop instance.

Module Configuration

In the begin of crud.php, we have the line

The models property must contain the name of every model that your module uses. This is not a PrestaShop requirement, but it is necessary to automatically create and repair the database. After that, we define the tabs that will be created in the backend menu.

With this definition, a tab name complete CRUD will be created in the backend menu, and will have the CRUD Model and Custom Operation children. The tabs are created using the addTab() method in the following code.

In line 17, we set the tab name for each language used in the shop. In line 25, we use the same addTab() function to create the children tabs. To remove the tabs, we use the following method.

With this basic methods, we may declare the model’s constructor, and install it.

The code above is self-explained, so we may continue to the install() method.

Firstly, we create the table associated to our model, if they didn’t exist yet. If they exist, but had some missing column, we create these missing columns in order to repair the database of the module. This is useful to fix errors that may occur after the upgrade to newer versions of the module.

Now, the following code displays the uninstall() method.

I really avoid to remove any database information when my modules are uninstalled, to avoid the losing of sensitive information. You may, instead, create a clean database method in the configuration page of your module, with the appropriated warning messages.

Model Creation

The model’s code is not that hard. We only need to detail the database configurations associated to the model (table name, column names/types, and so on). The following code may be found in the classes/CrudModel.php file.

The primary parameter is a column that will be assumed by the PrestaShop that is unique for each row in the database. Usually it will be the primary key of the table. The TYPE_INT, TYPE_STRING , TYPE_BOOL and TYPE_DATE constants are declared in the ObjectModel classvalidate refers to the name of some method of the Validation class, and will be automatically used to validate your model data before saving it in the database. Lastly, the db_type parameter required by CustomObjectModel for the database creation.

Controller Creation

As in the model, we does not have much work to do in the controller. Basically, we have to set the fields that will be displayed in our grid and the inputs that will be displayed in our create/update form. With this setup, PrestaShop will generate the grid and the forms automatically.

The controller must be placed in controllers/admin/AdminCrudModels.php. All the setup we need to do is made inside the class constructor, so we begin with the controller’s basic definition.

In line 7, we tell the controller to include the bootstrap files in the html of the page. The other parameters refers to our model table, primary key and class name. Now, we have to set the fields to be displayed in the list page.

The reader is invited to read the documentation of the HelperList class for more details of the code above. With line 14, the user will be able to change the status of our model by just clicking in the “v” or “x” icons. Also, the grid also gains the “Enable selected” and “Disable selected” bulk actions.

For the “Delete selected” bulk action, we need to insert the following code.

And to finish our list, let’s include some individual action buttons, as the following image.

This is easily achieved with the following code.

When the user clicks in the Edit button, he will see the following form, which will be automatically filled by PrestaShop.

He may also insert new register in the database, clicking in the “Insert new” button in the toolbar of the list.

(sorry, my shop is in pt-br)

To generate this form, we only have to add the following code to the controller’s constructor.

Again the reader is invited to check the documentation of the HelperForm class to understand the code above.

And this is everything we have to do to create a simple CRUD in the backend with PrestaShop. If you does not want so much customizations, this is really an easy task. However, if you wanna customize several elements of the page, you have to deeply check and understand what is behind the scenes. To do so, check the AdminControllerHelperList and HelperForm classes. They have almost all the code that is used to generate the pages that use used in this tutorial.

Using Multilanguage Fields

Multilanguage fields are fields that may have a different value for each language enabled in your shop. This is achieved with a table named <your_model_table>_lang, that contain a field named id_lang, a field named <your_model_primary_key> and any multi lang field of the model. This column and its fields will be created automatically by the CustomObjectModel class.

To configure a field as multilanguage in your model, you only have to modify the $definition property.

So, when you (re)install your module, the multilanguage table will be created automatically.

After the database is correctly created, we have to configure our controller in such a way that PrestaShop will allow us to easily create multilanguage records. This is achieved through the the $fields_form property of your controller.

Then, the form automatically created by PrestaShop will be like the following image.

Multi language fields created by the HelperForm class.

In the list action of your controller, PrestaShop will automatically display the value in the language that is configured by your user. With these two simple modifications, aided by the CustomObjectModel class, you have a fully functional multilanguage CRUD!

[PrestaShop] – Sending AJAX Requests to Admin Controllers

In this article I will talk about a problem that I had when developing my first PrestaShop module: how to do a AJAX request to my AdminControllers? Usually AJAX requests are not so difficult to be done, but things become more complicatet  when the target is a controller of the AdminController class.

Firstly, for the AJAX request to be done, there must have a record in the tab table related to the controller being targeted. You can create this tab whith the following code.

At line 3, if you set the tab as active, it will be displayed in the administration menu of the shop. If you are creating this controller just to send AJAX requests to it, leave active as 0. At line 9, tabname is the text that will be displayed in the administration menu, for each language that is installed in the shop. If you set active=0, this value is not important.

Now, your controller is already acessible through the url /adminpanel/index.php?controller=<controllername>. But all admin URLS must contain a security token for the request to reach the desired controller. For this token to be visible via Javascript, I usually set it as an attribute of some div in my template.

The controller responsible for generate the above template must set the $token$ variable with the security token.

Finally, we may do the AJAX request with the following code.

The code above will do a POST request to the method  ajaxProcess<ActionName> of the controller <controllername>.php. Notice that the method name is case sensitive. In the end, write this method in your controller.

Using Node.js to control hardware over the WEB with the Firmata protocol

In this article I will talk about a very interessing application that may be build using Node.js: the control of hardware over the WEB. The choosen application is the control of several leds connected to one Arduino board. The Node.js server will communicate with the Arduino board to set the status of the leds. The hardware layer is very simple, but is enough to illustrate the concepts that are necessary to build more complex applications. The full source-code of the application may be download from it’s GitHub repository.

The application

Firstly, I will describe the application in details. It consist of two leds that may be turned on and off by the user through a WEB interface. The number of leds is not important here. Also, the user may check the status of the leds at anytime. The same application may be used by several users. If one of them change the status of one led through the application, the other users will be notified in real time.

The application consist of three layers:

  • User Interface – Displays the status of the two leds, and allows the user to change the status of the leds. These states are kept updated with the physical state of the leds. This layer only communicates directly with the Node.js server;
  • Node.js server – Receives the leds’ status change requests, and send them to the Arduino board. This layer is also responsible for informing the user interface layer when the status of one led got changed. The Node.js server communicates with both the user interface layer and the hardware layer;
  • Hardware – Consists of the two leds more the Arduino board.

The communication between the user interface and the Node.js server will be done with the socket.io package, and the communication between the server and the Arduino board will be serial. These three layers will be detailed below.

User Interface

The user interface is very simple. Each one of the two leds is displayed as a icon, that may be black (led turned off) or yellow (led turned on). Also, there are two action buttons. If one led is turned on, and it’s correspondent button is pressed, this led will be turned off; otherwise, it will be turned on. The leds names correspond to the physical port of the Arduino board which it they are connected to.

User interface made up by two leds, each one with a correspondent action button.

To generate the HTML of the page, I used the template engine jade. It has a very simple language, which makes very easy to generate the HTML. I will not detail this engine, but the reader is invite do read it’s official documentation. The following code generate the HTML of the whole interface.

The css in style.css is very simple, so I will not insert it in this article to not let it unnecessarily big.

Now, I will describe the javascript code that will make the communication between the user interface and the Node.js server. This communication will be done with the socket.io package. This package keeps a connection between the browser and the server always open, so any of the two may send a message to the other anytime.

This connection is established with the following code.

In this code, localhost is the address of the Node.js server, and 5555 is the port through which the messages will be sent. Now, when a button is pressed, we need to inform the server. This will be reached triggering an event, which will be called toggle_led. 

In the line 3, the event toggle_led is triggered. This event will be handled by the Node.js server in real time, and will receive the led_number argument. This argument contains the number of the led which the state should be toggled. At last, we need to update the state of the leds in the interface when the server turns on of off some of them.

Now, the server will trigger the changed event, and will send the current state of the two leds as argument. So, each one of the two leds receive the class on of off, depending on whether they are on or off. Every user that are running the application will listen the changed event when the server change the state of one led. So, they all will have their interface updated with the physical state of the leds in real time.

Node.js server

Even though this is a simple application, I have choosen to use the Express framework, so that it can be improved to a more complex application with no much pain. Because I have already introduced this framework in other aticle, I will focus only in the communication between the server and the interface layer, and between the server and the Arduino board.

Firstly, add the packages socket.io and johnny-five to your application. The socket.io package was alread introduced, and the johnny-five is the package responsible for the communication with the Arduino board. The following code initialize the leds and the communication with the board.

In line 4, we set the serial port which the Arduino board is attached to. In line 7, we listen to the ready event of the board, which is triggered afer the communication is initialized. In the end, after the leds are initialized, the ioChanged() function will trigger the changed event to the inteface layer. This function will be described below.

To communicate in real time with the interface layer, we need to initialize the socket.io package.

So, when someone connects to the server through the 5555 port, the following event will be triggered.

Firstly we check if the communication between the server and the Arduino board has already been established. If so, we trigger the changed event just to send the updated state of the board to the new user. In line 5, we listen to the toggle_led event triggered by this user. In line 6, we toggle the status of the user requested by the user, and so, trigger the changed event to every user that is running the application.

Lastly, follows the ioChanged() function.

Hardware layer

There is not much to be done in the hardware layer for the communication to be stablished with the Node.js server. The johnny-five package, which is responsible for this, uses the Firmata protocol. So, we only need to use a program in the Arduino board that implements this protocol. Fortunately, the default Arduino IDE already has an example with this protocol implemented. We just have to record it in the board!

To do this, go to File -> Examples -> Firmata -> StandardFirmata, as displayed in the image below.

Now, just upload this program to the Arduino board, and the communication will work! You just have to connect the leds to the pins 2 and 13 of the Arduino, and the application is finished.

Node.js: creating a REST server using the Express Framework in a MVC Approach

Hello, it’s been a long time. I am without any free time because of my master’s degree project. In this post I will talk about a piece of a project that I an developing, in which I had to create a REST server with Node.js.

I am a big fan of CakePHP, because it is the first web framework that I used. Because of this, I always try to find similarities with CakePHP when I start developing a project or learning a language. So, I tried to keep the server structure to be the most like CakePHP as possible.

obs: if you are not familiar with the MVC pattern, read this introductory article.

In this project, the most of models methods use the Bluebird package for promises. Furthermore, I chose to use the MySQL database. If you want to use other database, some changes may be needed to the model layer.

The full source code of the server may be downloaded here.

Application Basic Structure

If you are already familiar with the MVC pattern, you will easily understand the directory structure. There is a directory for the controllers and one for the models, but no one for views. I chose to send data from the server only using JSON. If you want to support mode format, like XML, you may use an additional views layer, separated from the controllers.

Before running the server, edit the config/config.js file, with your database configurations and the port to be used by the server. You also may specify the port using the environment variable PORT.

Routes

The Express framework already have a very good routing system, so only a few work was needed here.

Each controller has a addRestRoutes method, which add the basic controller’s routes automatically. These routes map to actions in the Controller class. Each controller of your application must inherit the Controller methods.

The default routes are for an hypothetical BlogsController are

  • GET /blog/1: maps to findById
  • POST /blogs/searches: maps to search
  • POST /blogs: maps to create
  • PUT /blog/1: maps to edit
  • DELETE /blog/1: maps to delete

The search, create and edit actions require some parameters to be passed by the request body.

To the search resource, these parameters must be like the following.

The create and put parameters are only the data to be stored in the database.

If your controller must use some route out of this pattern, you may use the addCustomRoutesmethod.

Controllers

The basic structure of a controller is

The file must be named CarsController.js and to be inside the controllers folder.

Models

The models need some basic configurations in order to use the methods of the Model class.

The main methods inherited from the Model classes are listed below.

Model.find

Do a SELECT query on the database.

Usage

Parameters

  • conditions : array of conditions in the squel format
  • limit : maximum number of records to be returned
  • offset : number of records to be skipped
  • group : SQL group by clause
  • recursive : deep level to look inside the model’s associations

Model.findById

Search for a record with the specified id. Its only parameter is the id to search for. This method returns a rejected promise if there is no record with this id.

Model.save

Create or update an existing record.

Usage

Model.delete

Remove a record from the database. Its only parameter is the id of the record to be removed.

Squel: SQL Generator for Node.js

In this article, I will talk about the squel package, a SQL generator for node.js. To use a SQL generator is very productive, since we don’t need to directly write and debug the queries. Even though I will just show some simple queries, the package is very versatile.

Warning: You may also use squel on the browser, but you shouldn’t run any query in the server that was generated on the client, since it has serious security issues!

Installing and Configuring

In order to install and automatically add squel to your package.json dependencies, run

This package doesn’t need and configuration to be used.

Basic Usage

I took some examples from the official documentation.

Insert

Select

Update

Delete

Using Prepared Statements

Whenever is possible, you should use prepared statements. I wrote an article which shows how to use prepared statements with the `node-mysql2´ package, when you write your queries by yourself.

To do it with the squel package is simple.

And there is much more. It is possible to run subqueries, complex where conditions and generate queries specific to mysql or postgres databases. You really should take a look at the complete documentation!

Caching your views with Redis using the Express Framework

Hi again! On this post, I will show how to use the express-redis-cache package to cache your views, decreasing your page load.

The package is very simple to use, but he is also very complete. I am using it on this blog, and am not missing any important feature. It’s also very efficient, because it uses Redis to store the cache.

Installing and Configuring

To install, include express-redis-cache to your package.json dependencies, or type

I will assume that you are using the 0.1.8 version, which is the most recent at this moment.

So, we must specify the basic configurations of the package.

The prefix option is used to construct the strings that will identify each one of the cached pages. The app object is exported like a module, so we can use the same cache object in other files, with the same Redis connection.

If you is already conected to a Redis server, you can use the option {client : redis_client_object}, omitting host, port and auth_pass. If your Redis server require a password, you need to authenticate the redis client by yourself.

Basic Usage

The cache.route() middleware must be used on every route that should be cached.

This will ask to cache to check if there is some cache named APP_ NAME + '/'. If not, thehome() function will be executed, and ‘Welcome to my app’ will be automatically cached, with the APP_NAME + '/' name. While it doesn’t expire, the home() function will no longer be called.

Changing the Entries Name

You can change the name of a entry using

or

or

Choosing to Cache (or not) a Page

You can chose not to use the cached answer on the fly.

Ovewriting the expire option

To change the expiration time of a single entry, do

Excluding Cache Entries

You can exclude cache entries, even if they haven’t expired yet.

If you have pagination at your homepage, with each page saved on entries like home.page.0,home.page.1, and so on, you can exclude all these entries with a single call to del() function.

Debugging

To listen the various debug messages sent by the package,

In medium/large applications, your page load will decrease considerably if you cache your views. However, you must be careful to clear the cache whenever your views are updated. You don’t want an old HTML including a updated javascript, do you?

Generating a Blog Sitemap with Node.js using the Express Framework

In this article I will talk (superficially) about what sitemaps are, and how to generate them with an existing Node.js package. Even though the practical examples are focused on blog applications, they may easily be extended to a more general website. The sitemap will be cached by 24 hours (the time may easily be changed).

I chose to use the sitemap package, which is not specific to Express, but is very easy to use, and has an interesting cache system. You may take a look at the express-sitemap package, specific to Express, but the only advantage that I found in it is that it can automatically add static pages of your app object to the sitemap. If you have lots of static pages, this may be useful for you. For a blog, I haven’t found any reason to use it instead of the more generalsitemap package.

If you know the basic about sitemaps, you may skip the first section of this article.

An Introduction to Sitemaps

Sitemap is a protocol made for helping search engines on pages indexing. It may be a single text file, on which each line is the URL of a page you want it to be indexed.

Naturally, if you don’t include some URL on your sitemap, this page may still be indexed. If you don’t want some page to be indexed, read about robots.txt. Likewise, if you add a page on the sitemap, it may not be displayed on the first page of any search engine.

Sitemaps are just tools to help the search engines to find every page of your site. Optionally, you may add a relative priority to every page of your site, if you think that some pages are more important than others. This priority is relative only to pages of your own site, and doesn’t have any effect to make your site be better ranked than other sites.

The most common sitemap format is the XML.

Here,the http://www.example.com/ was updated for the last time on 01-01-2015, is expected to be updated monthly, and has a 0.8 priority (in a scale of 0 to 1). The default priority is 0.5.

Installing and Configuring the Sitemap Package

To install the package, include sitemap to the dependencies of your package.json file, or use the npm.

Now create a routessitemap.js file and add it to your app object.

So, you can generate the sitemap

Visit yoursite.com/sitemap.xml, and there it is!

Including your Posts on the Sitemap

Let’s get the URLs of yours posts, so we can include the on the sitemap. You don’t need to addevery page of your site to the sitemap, only the ones which offers some new content. Pagination or search pages, usually, don’t need to be included.

For this reason, we will add to the sitemap only the URLs of the posts.

The sm.toXML() method checks if the XML needs to be generated again, or if the cache is still valid. If your sitemap have lots of pages, this will result on a great performance gain.

If you prefer to use some package specific to Express, you may use the express-sitemap with only a few modifications. The most important is that the express-sitemap doesn’t have any type of cache. To avoid searching every post from the database, and creating the XML every time the sitemap is requested, you can add a cron job to generate the XML at regular intervals, or save the last time a sitemap was generated in a variable, and check for the expiration by yourself.

Using Node.js with MySQL

Hi there! In this post, we shall talk about the use of MySQL database with Node.js. I will use the node-mysql2 driver. Unfortunately, the driver doesn’t support promises. If you want to use promises instead of callbacks, you may use bluebird promisification.

Installation

To install node-mysql, runs the command

You may also install adding the mysql2 package to the dependencies list of yourpackage.json file. To more details, check package.json;

Configuration

Create a config/database.js file, with the convenient adjustments.

The supportBigNumbers options must be true if any of your tables has a BIGINT or DECIMALcolumn.

Check the full list with the possible configurations here.

Basic Usage

In order to stablish a database connection, use the following code.

So, you can easily run queries

The end() method close the connection, waiting for every pending queries to be executed before this.

Prepared Statements

To avoid SQL Injection attacks, you should use prepared statements whenever you make use of user provided data. Furthermore, prepared statements runs very quickly when similar queries are executed successively.

You can also prepare the statement by yourself.

Connection Pool

The driver also supports connection pool. To do so, add to your config/database.js file the line

This limits the maximum of simultaneous connections to the database.

Now, we may get connections from the pool.

Transactions

It’s very simple to use transactions.

No error handling has been done for readability reasons. Note that this functions just runs theSTART TRANSACTION, COMMIT and ROLLBACK commands, and some queries may cause implicit commits.

Using Promises

Unfortunately this package doesn’t natively support promises. However, you may use thepromisify() and promisifyAll of the bluebird package.

Jade: Creating reusable blocks witn Mixins

he benefits of modularity have been widely discussed on software engineering, are not the focus of this article. Here, i am only concerned with the implementation aspects. If you are interested, check also the inheritance section on the Jade official documentation.

The examples used on this article are taken from the official documentation.

Mixins

Mixins are reusable code blocks, which help us to make our views clean and organized. Firstly, let’s see a simple example which illustrates the mixins’ declaration and usage.

The following Jade code

generates the HTML

The list mixin may be used on any jade file, since the file on which is is declared has been properly included.

Arguments

Now, let’s see a more useful example, on which the mixin takes arguments.

produces the HTML

Blocks

Also, mixins may receive blocks, in order to produce more customized HTML.

will generate the HTML

Named Attributes

You may use named attributes too.

producing the HTML

The named attributes are automatically escaped when calling the function. So, the must be used on the mixin with the ! operator, or the attribute will be escaped twice.

Unknown Arguments Number

Lastly, mixins may receive an unknown number of arguments. The following Jade code

produces the HTML