• magento-performance-optimization

Why Optimize website for speed?

  • Recent research says that more than 60% online shoppers will skip if site pages are slow.
  • A happy visitor probability of revisiting site is much higher.
 

Why it is more so in case of Magento®?

  • Magento® has lots of features and flexibility. To get features Magento® uses EAV and other concepts which make Magento® a heavy site.
  • To overcome these short comings Magento® team has recommended and applied additional concepts like indexing, caching, and FPC. But to achieve this, you need to optimize Magento® for performance (Magento® Speed Optimization).
 

Why to choose oscprofessionals for this service?

  • We have experience of optimizing lots of Magento® sites on both platform (Enterprise and Community) .We have developed tools that let us analyze bottleneck areas. This in turn means we will be on target and take lesser time to optimize.
  • We deliver quality while being economical plus we visualize all aspects of cart while optimizing.
  • We will give support related to our work for 6 months after work has been delivered.
  • We have Certified Developers to customize, optimize Magento®.
 

Magento® Speed Optimization process at Oscprofessionals:

Analysis:

  • We use tools like webpagetest and GT Matrix to get idea on total page download time and different aspects that will effect download time.

Implementation:

  • Implementation to reduce PHP execution time by targeting use of cache and removing unwanted blocks. This will result in reduction of TTFB.
  • Implementation to reduce total download time, here we improve grades/score as per webpagetest/GT Matrix report.
  • This will result in reduction of total document download time.
    Optimization related Server activity:
  • If server is dedicated & we are provided access, we will modify server setting to improve/boost execution speed.
  • e.g: Expire header, Keep alive etc.
    Also we provides & communicate with hosting company for server related configuration.
    Links on How to Speed-Up Magento® Sites:

Oscprofessionals blog series for Magento® Speed Optimization
Important URL on Stack Overflow for Magento® Speed Optimization

 

Tools recommended to check speed:

GT Matrix

 

Configuration tips for webmaster related to Magento® Performance Optimization:

  • Flat catalog to be switched on.
  • Remove customer group not in use.
  • Remove dummy store if not in use.
  • Delete attributes created but not used later.
  • Make sure that attributes configuration is done optimally.
  • Cache should be always on.
  • Compiler mode to be set on.
  • Merge JS and CSS.
  • Delete Magento® cart rules if not in use.
  • Enable Cache ( If some FPC installed make sure its enabled).
  • Delete additional stores if not in use.
  • Delete cart rules if not in use.
  • Minimize number of searchable attributes as these result in columns in flat catalog table and will slow down your search.
  • Use Solr search if module installed.
  • Number of days log should be minimized.
  • Switch off Logs writing in Magento® ( In most cases not recommended as tracking issues might be difficult ).
  • Remove attributes if not needed. With proper care we can even remove some system attributes if not in use.
  • Optimization by proper attribute management like setting required attribute to yes or is search-able or required in listing etc.
  • Your cron should be run in off-peak hours.
  • Remove staging stores if on EE.
  • Remove Admin notification module.
 

Recommended Server Setting for Optimization:

Use Nginx Server instead of Apache.

  • PHP Version higher than 5.4
  • APC, Redis recommended for caching.
  • MySQL Version higher or atleast 5.5 (Percona Recommended).
  • CDN for Images.
  • Split DB and Webserver.
  • Run Admin, Cronjob from a separate server.
  • Set Expiration Header.
  • Enable gzip Compression.
  • Enable keep alive (will consume a little bit extra memory but will save some time).
  • Use query cache but avoid size greater then 64 MB.
  • Session to be on Ram.
  • Reduce hard disc file reads and try reads from ram as this is faster.
  • Upgrade PHP version to above 5.3
 

Recommended Tips to reduce page download time:

  • Optimize Images.
  • Use Image Sprites.
  • Image Lazy Loading.
  • Expire Header to be set.
  • Reduce number of DOM element.
  • Avoid or reduce number of 3rd party request.
  • Use of CDN.
  • Merge CSS and JS.
  • Enable compression in .htaccess (mod_gzip for Apache 1.3, mod_deflate for Apache 2).
  • Make your output W3C compliant.
  • Reduce DNS Lookups : Minimize third party scripts if needed execute them as late as possible.
  • Reduce server response time( TTFB).
  • Minimize HTTP Requests.
  • Minify CSS.
  • Prioritize above-the-fold content.
  • Put Style sheets at the Top.
  • Put Scripts at the Bottom.
  • Avoid CSS Expressions.
  • Avoid Redirects.
  • Remove Duplicate Scripts.
  • Reduce the Number of DOM Elements.
 

Modules Recommended for Magento® Speed Optimization:

  • FPC.
  • Fooman Speedster.

Developers Action and Precaution for Magento® Speed Optimization:

  • Use Ajax where ever possible.
  • Remove all modules not in use by removing there XML.
  • Just disabling will not do.
  • CMS blocks take more time then a Magento® block so unless you want a block to be modified do not use CMS blocks.
  • Do not use collection count use collection getSize to get what is the collection size.
  • Use of getChildHtml (‘childName’) is recommended as this will cache block against direct use of block code in .phtml file.
  • If on EE make sure that most pages are delivered without application initialization. Even if one container needs application initialization its going to effect execution speed as apart form URL rewrites most of the other code will get executed.
  • Check XML for blocks placed in default handle and if those blocks not on specific page then move those XML values from default handle to specific handles. It has been observed that lots of blocks are executed that are not displayeDelivering images depending on device in use.d.
  • If using FPC make sure your containers are cached and repeat request for container is delivered via cache. Improper placeholder definition results in container cache not being used but each time new container content getting generated.
  • Analyze page blocks and variables and if possible add those variables/blocks to cache.
  • Write your own solr search populate to bypass Magento® search indexing.
  • Clean _cl tables or reduce _cl table rows.
  • Some observers are not required for all stores so in case those observers are not applicable to a specific Magento® site then they should be removed.
  • Make sure FPC is applicable to most of the site pages. Specially when you added some new controllers to delivering a page.
  • Magento® has lots of features. For this it has many events and associated observers. There are few features that are not used by a store so any observer related to that feature should be removed. e.g : If you check enterprise version there is category permission concept which if not used, then its recommended that on save after events permission related observers to be removed.
  • If a specific attribute is to be saved for a product then instead of call $product->save call a function that will save specific attribute.
  • In EE version that has partial indexing and triggers modify triggers to avoid multiple entries to_cl tables.
  • No .phtml files bypasses blocks and use modules or resources directly. As this will result in no caching which inturn means more work for Magento®.
 

Is your Magento® site performance ok?Check following points?

  • TTFB less than 1.2 Seconds (VPS or Shared Server).
  • TTFB less than 0.8 Second (Dedicated Server).
  • TTFB less than 0.6 Second (FPC Enabled + Dedicated Server).
  • Page download less than 5-6 Seconds for (VPS or Shared Server).
  • Page download less than 5 Seconds for (Dedicated Server).
  • Page download less than 5 Seconds for (FPC Enabled + Dedicated Server).
  • Number of request less than 60.
 

Magento® Speed Optimization Admin Configuration Related Screenshots

1] Enable Flat Categories and Products:

There are following two options which you can enable or disable in the Magento® backend.

a) Flat Catalog Category

b) Flat Catalog Product

It is recommended that you enable this feature.

2] Enable Magento® Cache:

Go to Magento® admin: System > Cache Management

3] Combine JS and CSS in magento®:

Go to magento® admin : System > Configuration > ADVANCED > Developer

4]Log Cleaning Setting:

When your site get high traffic, this cause a big trouble.

In all cases, you should config your Magento® Log cleaning.

5] Disable Magento® logging:

Disable the Magento® Log: System -> Configuration -> Advanced -> Developer -> Log Settings (default is disabled).

 

Recommended tips to improve Magento® Speed Optimization:

  • No.phtml files bypasses blocks and use modules or resources directly.
    As this will result in no caching which inturn means more work for Maegento.
  • Delivering images depending on device in use.
  • Some of the FPC recommended for community :
    Lesti( Free as on date),
    Amasty(commercial),
    extender(commercial)
    and Bolt(commercial).
  • Warming Cache.
  • Controlling bots by .htaccess during peak hrs.
  • Pre-populating values in a custom table for Layered
    Navigation via a custom script that executes daily by cron.
  • Making sure to avoid unwanted Keys to reduce cache size.
  • Using a higher PHP version 5.4+
  • Using a higher Mysql version( 5.5 +)
  • Reduce number of Dom elements.
  • Move all js out from html pages.
  • Remove commented html.
  • Attributes sets to be properly created, So as to avoid unwanted attributes
    getting associated with products.
  • If there are very few attributes for Layered Navigation then those
    should be populated in Flat Table, and Layered
  • Navigation related collection code to be modified to Query Flat
    Table for filter result.
 

Magento® Speed Optimization points Technical Explanation:

Flat catalog to be switched on:

  • Magento® uses EAV to allow for adding many attributes.
  • Magento® Indexes attributes to populate flat table.
  • Magento® uses flat table for listing pages to reduce SQL Queries.
    When flat disable:

This is a Heavy Query

SQL Query for listing page : SELECT `e`.*, `cat_index`.
`position` AS `cat_index_position`, `perm`.
`grant_catalog_category_view`,
`perm`.
`grant_catalog_product_price`,
`perm`.
`grant_checkout_items`,
`price_index`.`price`,
`price_index`.`tax_class_id`,
`price_index`.
`final_price`,
IF(price_index.tier_price IS NOT NULL,
LEAST
(price_index.min_price,
price_index.tier_price),
price_index.min_price)
AS `minimal_price`,
`price_index`.
`min_price`, `price_index`.
`max_price`,
`price_index`.`tier_price`,
IF(( SELECT vm.
view_mode AS
required_options FROM
catalog_product_option
AS cpo LEFT JOIN
custom_options_option_
view_mode AS vm ON cpo.
option_id=vm.option_id AND
(vm.store_id=’1′ OR vm.store_id=0)
WHERE e.
entity_id=cpo.product_id
AND cpo.is_require=1
ORDER BY vm.store_id DESC LIMIT 0,1)=’1′,’1′,’0′)
AS `required_options`
FROM `catalog_product_entity`
AS `e` INNER JOIN
`catalog_category_product_index` AS `cat_index`
ON cat_index.product_id=e.
entity_id AND cat_index.
store_id=1 AND cat_index.visibility
IN(2, 4) AND cat_index.
category_id = ’18’ LEFT JOIN
`enterprise_catalogpermissions_index`
AS `perm`
ON perm.category_id=cat_index.
category_id AND perm.
customer_group_id= 0 AND perm.
website_id=1
INNER JOIN `catalog_product_index_price` AS
`price_index` ON price_index.
entity_id = e.entity_id
AND price_index.website_id = ‘1’ AND
price_index.customer_group_id = 0 WHERE
(perm.grant_catalog_category_view != -2 OR perm.
grant_catalog_category_view IS NULL) AND
(cat_index.store_id=1) ORDER BY
`cat_index`.
`position` ASC LIMIT 12

When flat enable:
This is Comparatively a Lighter Query

SQL Query for listing page : SELECT 1 AS
`status`,
`e`.`entity_id`, `e`.`type_id`,
`e`.`attribute_set_id`, `cat_index`.`position` AS
`cat_index_position`,
`perm`.`grant_catalog_category_view`, `perm`.
`grant_catalog_product_price`,
`perm`.`grant_checkout_items`, `e`.`name`,
`e`.
`short_description`, `e`.`price`,
`e`.`special_price`, `e`.`special_from_date`,
`e`.
`special_to_date`, `e`.`small_image`,
`e`.`thumbnail`, `e`.`news_from_date`,
`e`.
`news_to_date`,
`e`.`url_key`, `e`.`required_options`,
`e`.`image_label`, `e`.`small_image_label`,
`e`.
`thumbnail_label`, `e`.`msrp_enabled`,
`e`.`msrp_display_actual_price_type`, `e`.
`msrp`, `e`.
`tax_class_id`, `e`.`price_type`,
`e`.`weight_type`, `e`.`price_view`, `e`.
`shipment_type`,
`e`.`links_purchased_separately`,
`e`.`links_exist`, `e`.`giftcard_amounts`, `e`.
`allow_open_amount`, `e`.`open_amount_min`,
`e`.`open_amount_max`, `price_index`.`price`,
`price_index`.`tax_class_id`, `price_index`.
`final_price`, IF(price_index.tier_price IS NOT NULL,
LEAST(price_index.min_price, price_index.tier_price),
price_index.min_price) AS `minimal_price`,
`price_index`.`min_price`, `price_index`.
`max_price`,`price_index`.`tier_price`,
IF(( SELECT vm.view_mode AS required_options
FROMcatalog_product_option AS cpo LEFT JOIN
custom_options_option_view_mode AS vm
ON cpo.option_id=vm.option_id AND
(vm.store_id=’1′ OR vm.store_id=0) WHERE
e.entity_id=cpo.product_id AND cpo.is_require=1
ORDER BY vm.store_id DESC LIMIT 0,1)=’1′,’1′,’0′)
AS `required_options` FROM
`catalog_product_flat_1` AS `e` INNER JOIN
`catalog_category_product_index` AS `cat_index`
ON cat_index.product_id=e.entity_id AND cat_index.
store_id=1 AND cat_index.visibility
IN(2, 4) AND cat_index.category_id = ’18’ LEFT JOIN
`enterprise_catalogpermissions_index` AS `perm`
ON perm.category_id=cat_index.category_id AND perm.
customer_group_id= 0 AND perm.website_id=1
INNER JOIN
`catalog_product_index_price` AS `price_index`
ON price_index.
entity_id = e.entity_id AND price_index.
website_id = ‘1’ AND price_index.customer_group_id = 0
WHERE (perm.grant_catalog_category_view != -2 OR perm.
grant_catalog_category_view IS NULL) AND
(cat_index.store_id=1) ORDER BY `cat_index_position
` ASC, `cat_index`.`position` ASC LIMIT 12

 

Proper Handling of Attributes:

Make sure that attributes configuration is done optimally and Delete attributes created but not used later:

  • Configure your attributes to match website needs.

For Example:

Catalog -> Attributes -> Manage Attributes -> Frontend Properties

Suppose, attribute not used in product listing page but in above configuration Used in product listing is ‘Yes’ then attribute column created in flat table, which will result in additional resources required during indexing and more columns in flat table.Set only those attribute frontend properties to ‘Yes’ that you’re actually going to use. Set all other to ‘No’. Don’t use in quick search, advanced search compare.

Merge JS and CSS:

After merging of JS and CSS, we can observe that number of CSS & JS file are reduced. This will result in lower number of HTTP request hence lesser document download time.

  • Compiler mode to be set on:
  • The Magento® compilation feature limits the number of directories that PHP has to search through when looking for PHP-files.
  • Compiler on limits the resulting in reduction of exection time.
  • Give you a 30%-50% performance : System > Tools > Compilation.
    Use Solr search (if module installed or install solr module if server supports)
  • Offload the layered navigation and category page results rendering from mysql to Solr.
  • Instead of Querying MySQL Solar is Queried resulting in reduced MySQL load.
  • Will result in lower indexing time.
    Remove customer group not in use:
  • Yes, remove customer group not in use . Reason is that, number of entries are added in tables related to customer group.
  • Example:catalog_product_index_price,catalog_product_index_group_price,
    catalogrule_customer_group, catalogrule_group_website,
    catalogrule_product and in any more tables data are present related to customer group as per different variations.

    Cache should be always on:
  • Switched on all the avaliable caches in the Magento® admin panel.
  • When cache is on result fetched from cache so lots of SQL queries reduced.
  • System -> Cache Management
    Delete additional stores if not in use:
  • Each Stores results in additional flat tables.
  • Each Stores needs separate indexing which will result in higher indexing time.
  • Each Stores results in more Database table row entries which inturn will make SQL queries take more time.
  • Eg. Store 1 : catalog_product_flat_1Store 2 : catalog_product_flat_2

    Some important links :

    http://www.oscprofessionals.com/blog/understanding-full-page-cache-concept-magento
    and
    http://dionbeetson.blogspot.com.au/2014/11/magento-performance-tips-for-scalability.html

One stop delivery for Magento® Development and Customization Services in Australia, United Kingdom, UAE, United States. Please get in touch for a free quote on quality and affordable development services irrespective of geographic factors.

Browse Services

 
* Indicates required field

Join us to know your Websites Performance

* Indicates required field

Terminologies used in Optimization

Ajax (Asynchronous JavaScript and XML) :
With Ajax, web applications can send data to and retrieve from a server asynchronously (in the background) without interfering with the display and behavior of the existing page. Ajax is not a single technology, but a group of technologies. HTML and CSS can be used in combination to mark up and style information.

Auto Scaling :
Auto scaling is a cloud computing feature that allows users to automatically scale cloud services, like virtual machines (VM) and server capacities, up or down, depending on defined situations. Cloud computing providers, such as Amazon Web Services (AWS), offer this feature.

Apache .htaccess or Nginx Virtual Host Optimization :
By default, the Apache web server is working in conjunction with nginx. The benefits are that web pages load faster and server resources are saved. To learn how Apache and nginx collaborate by default, see Apache with nginx.

Caching :
The storage of web files on a computer or server so they can be accessed quicker by the end user.

Cache Flushing :
Other than security, cache flushes have a variety of effects on computer performance. Disk write caching can be used to increase performance on computers that often write information to the hard disk.

Compress Images :
The objective of image compression is to reduce irrelevance and redundancy of the image data in order to be able to store or transmit data in an efficient form.

CSS and JavaScript Compressing :
Compression or minification, is the process of removing all unnecessary characters from source code without changing its functionality. These unnecessary characters usually include white space characters, new line characters, comments, and sometimes block delimiters, which are used to add readability to the code but are not required for it to execute.

Database Checks :
A check constraint is a type of integrity constraint in SQL which specifies a requirement that must be met by each row in a database table. The constraint must be a predicate. It can refer to a single or multiple columns of the table.

Document Download Time :
Download refers to the process of receiving data over the Internet. Lesser the time taken to download, better the site performance.

Document Object Model (DOM) :
The Document Object Model, or DOM, is an interface to allow programs and scripts to update content, structure, and style of documents dynamically.
It is platform- and language-neutral.
The DOM is the API that binds JavaScript and other scripting languages together with HTML and other markup languages. It is what allows Dynamic HTML to be dynamic.

Database Tables and Magento® Logs Clean-up :
Magento® maintains several tables for logging. These tables log things such as customer accesses and frequently-compared products. Maintaining the database through log cleaning can result in a dramatic improvement in site performance and latency.

Flat Catalog :
Flat catalog options not only enable the flat catalog and flat product indexers but also enable indexing for catalog and shopping cart price rules. If you have a large number of SKUs (500,000 or more), Magento® can index catalog and shopping cart price rules quickly if you enable the flat catalog product option.

Full Page Caching :
Full page caching involves the storage of the full output of a page in a cache, so that in subsequent page loads will not require much server load at all. For high-traffic Enterprise level eCommerce sites, full page caching is imperative in order to keep server load as low as possible, and to avoid downtime during periods of high traffic.

Google Page Speed Tools :
Page Speed Insights measures the performance of a page for mobile devices and desktop devices. It fetches the URL twice, once with a mobile user-agent, and once with a desktop-user agent.
The Page Speed Score ranges from 0 to 100 points. A higher score is better and a score of 85 or above indicates that the page is performing well.

GTmetrix :
The GTmetrix API offers developers an easy way to utilize GTmetrix’s performance testing service. Using the GTmetrix API, you can integrate performance testing into your development environment or into your application.

Hole Punching :
Full page caching is a way for Magento® to load content for a user without having to fully initialize the application(which is less dynamic).
Magento® has built-in ways to achieve a more dynamic page through a process called “hole-punching.”

Indexing :
Indexing is the adjustment of the weights of assets in an investment portfolio so that its performance matches that of an index.

Lazy Loading :
Lazy loading is a design pattern commonly used in computer programming to defer initialization of an object until the point at which it is needed. It can contribute to efficiency in the program’s operation if properly and appropriately used. The opposite of lazy loading is eager loading.

Loading :
Loading is a design pattern commonly used in computer programming to defer initialization of an object until the point at which it is needed. It can contribute to efficiency in the program’s operation if properly and appropriately used.

Load Balancing :
Load balancing distributes workloads across multiple computing resources, such as computers, computer cluster, network links, central processing units or disk drives. Load balancing aims to optimize resource use, maximize throughput, minimize response time, and avoid overload of any single resource. Using multiple components with load balancing instead of a single component may increase reliability and availability through redundancy. Load balancing usually involves dedicated software or hardware, such as a multi-layer switch or a Domain Name System server process.

Minimizing HTTP Requests :
By combining external files and optionally including CSS and JavaScript directly within your XHTML pages, you can minimize the number of HTTP requests required to render your page. Each unique HTTP request requires a round trip to a server, introducing indeterminate delays. Users attune to fast, consistent response times. The more HTTP requests that your pages require, the slower and less consistent their response time will be.

Memcached :
Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.

Magento® Full Page Cache :
Magegiant Full Page Cache will reduce the waiting time of customers and it is less than 0.3 seconds instead of 3-5 seconds or longer. Your website speeds up lead to reduce your abandonment, increase your conversion and raise your own revenue.

MySQL Indexing :
A database index is a data structure that improves the speed of operations in a table. Indexes can be created using one or more columns, providing the basis for both rapid random lookups and efficient ordering of access to records.

MySQL Server Tuning :
A well tuned MySQL server can perform 2 – 3 x better than a poorly tuned MySQL server.
The default configuration file for MySQL is intended not to use many resources, because its a general purpose sort of a configuration file. The configuration file need to be customized and tuned if you intend on using complex queries and when you have good amount of data.

PHP Tuning and Configuration :
A well configured PHP environment is very important. In order to get maximum performance,

  • Use the latest stable PHP version. Major releases of PHP may bring significant performance improvements.
  • Enable byte-code caching with Op-cache or APC(Alternative PHP Cach ) . Byte-code caching avoids the time spent in parsing and including PHP scripts for every incoming request.
  • Tune realpath( ) cache.

Render Blocking :
Render means loading, so if something is render-blocking, it means that it is keeping the page from loading as quickly as it could.

Redis :
Redis is an open source key-value database sponsored by Pivotal. Data in a key-value database has two parts: the key and the value. Because Redis can accept keys in a wide range of formats, operations can be executed on the server and reduce the client’s workload.

Relational Database Service :
Relational Database Service is a stable, reliable, elastic and high-performance online database service based on AliCloud’s distributed system, Apsara. Relational Database Service supports MySQL, SQL Server, PostgreSQL and PPAS and provides a comprehensive set of features including disaster recovery, data back-up, monitoring and migration which helps reduce the resources you need to spend on database operation and maintenance.

Sprite Images :
An image sprite is a collection of images put into a single image. A web page with many images can take a long time to load and generates multiple server requests. Using image sprites will reduce the number of server requests and save bandwidth.

Speed Testing :
A test to work out the number of problems a person can solve in a set period of time. A test with time constraints. A test where speed will contribute to the final score. Compare power test.

Session Storage :
Session Storage is similar to local Storage, the only difference is while data stored in local Storage has no expiration set, data stored in session. Storage gets cleared when the page session ends. A page session lasts for as long as the browser is open and survives over page reloads and restores. Opening a page in a new tab or window will cause a new session to be initiated, which differs from how session cookies work.

TTFB :
Time To First Byte (TTFB) is a measurement used as an indication of the responsiveness of a web server or other network resource. TTFB measures the duration from the user or client making an HTTP request to the first byte of the page being received by the client’s browser.

Varnish :
Varnish is an HTTP accelerator designed for content-heavy dynamic web sites as well as heavily consumed APIs. In contrast to other web accelerators, such as Squid, which began life as a client-side cache, or Apache and nginx, which are primarily origin servers, Varnish was designed as an HTTP accelerator. Varnish is focused exclusively on HTTP, unlike other proxy servers that often support FTP, SMTP and other network protocols.

Web Page Test :
Complete testing of a web-based system before going live can help address issues before the system is revealed to the public. Issues such as the security of the web application, the basic functionality of the site, its accessibility to handicapped users and fully able users, its ability to adapt to the multitude of desktops, devices, and operating systems, as well as readiness for expected traffic and number of users and the ability to survive a massive spike in user traffic, both of which are related to load testing.

Web Speed Test And Score :
For Google, that means pages should load in less than a second. Slow sites are likely to pay the price in terms of lower search ranking, so it’s important to find out how fast your site loads and where the bottlenecks are.

W3C Compliance :
Standards-compliance is the compliance of a website or web browser with the web standards of the World Wide Web Consortium (W3C). To ensure inter-operability a standards-compliant web site does not use proprietary software methods or features of a browser.