Sohaib's Tech Blog

Something for every one

Magento 2.12 Varnish 5.0 Error 503 Backend fetch failed **SOLVED** PHP 7.1 Nginx Varnish 5.x — January 26, 2018

Magento 2.12 Varnish 5.0 Error 503 Backend fetch failed **SOLVED** PHP 7.1 Nginx Varnish 5.x

Magento 2.x 503 backend fetch failed varnish

Happy New Year – 2018

This blog is my diary so I can keep track what I fix and what I’ve learned and today after spending some time and pulling my hair, as we are about to launch our e-commerce site I was able to fix this error – This is a bug which is already reported a few months back. This will work with NGINX – Varnish 4.x or Varnish 5.x

/etc/varnish/default.vcl

Look for

.url = “/pub/health_check.php”;

Change the above line to

.url = “/health_check.php”;

On NGINX you need to fix the configuration file in my case it’s called nginx.conf.sample which is also inside Magento folder.

Line looks something like this before the modification

# PHP entry point for main application -location ~ (index|get|static|report|404|503)\.php$ {

Make changes to the following:-

# PHP entry point for main application
location ~ (index|get|static|report|404|503|health_check)\.php$ {

It’s a bug which was also reported on Github 

Don’t need your donation a simple THANK You comment is more than enough for me. 

Magento2 Authorize.NET CIM Opensource — November 16, 2017
Set up Magento 2 with Redis, Varnish and Nginx as SSL termination —

Set up Magento 2 with Redis, Varnish and Nginx as SSL termination

In this article, we will show you how to install Magento 2 on an Ubuntu 16.04 VPS with MariaDB, PHP-FPM 7.0, Varnish as a full page cache, Nginx as SSL termination and Redis for session storage and page caching.  This guide should work on other Linux VPS systems as well but was tested and written for an Ubuntu 16.04 VPS.

Login to your VPS via SSH

ssh my_sudo_user@my_server

Update the system and install necessary packages

sudo apt-get update && sudo apt-get -y upgrade
sudo apt-get -y install curl nano git

Install MariaDB 10.0

Install the latest MariaDB 10.0 server from the official Ubuntu repositories:

sudo apt-get install -y mariadb-server

When the installation is complete, run the following command to secure your installation:

mysql_secure_installation

Next, we need to create a database for our Magento installation.

mysql -uroot -p
MariaDB [(none)]> CREATE DATABASE magento;
MariaDB [(none)]> GRANT ALL PRIVILEGES ON magento.* TO 'magento'@'localhost' IDENTIFIED BY 'my_strong_password';
MariaDB [(none)]> FLUSH PRIVILEGES;
MariaDB [(none)]> \q

Install PHP 7.0, composer and all required PHP modules

To install the latest stable version of PHP 7.0 and all necessary modules, run:

sudo apt-get -y install php-fpm php-cli php-gd php-imagick php-mysql php-mcrypt php-pear php-curl php-intl php-xsl php-zip php-mbstring

Change few default PHP settings:

sudo sed -i "s/memory_limit = .*/memory_limit = 256M/" /etc/php/7.0/fpm/php.ini
sudo sed -i "s/upload_max_filesize = .*/upload_max_filesize = 128M/" /etc/php/7.0/fpm/php.ini
sudo sed -i "s/zlib.output_compression = .*/zlib.output_compression = on/" /etc/php/7.0/fpm/php.ini
sudo sed -i "s/max_execution_time = .*/max_execution_time = 18000/" /etc/php/7.0/fpm/php.ini

Composer is a dependency manager for PHP with which you can install packages. Composer will pull in all the required libraries and dependencies you need for your project.

curl -sS https://getcomposer.org/installer | php
sudo mv composer.phar /usr/local/bin/composer

Install Magento 2 from Github

Clone the Magento repository to the ~/myMagentoSite.com directory using the following command:

sudo git clone https://github.com/magento/magento2.git /var/www/myMagentoSite.com

Get the latest stable release, at the time of the writing it’s Magento 2.1.2:

cd /var/www/myMagentoSite.com
sudo git checkout $(git describe --tags $(git rev-list --tags --max-count=1))

Run composer to install all Magento dependencies:

sudo composer install

To continue with the installation you can either use the installation wizard or the command line, in this guide we will use the latter.

sudo bin/magento setup:install \
--base-url=http://myMagentoSite.com/ \
--db-host=localhost \
--db-name=magento \
--db-user=magento \
--db-password=my_strong_password \
--admin-firstname=First  \
--admin-lastname=Last \
--admin-email=user@myMagentoSite.com \
--admin-user=admin \
--admin-password=my_strong_password123 \
--language=en_US \
--currency=USD \
--timezone=America/Chicago \
--use-rewrites=1

If the installation is successful you will see something like below:

[SUCCESS]: Magento installation complete.
[SUCCESS]: Magento Admin URI: /admin_mejj1n

Run the crontab command to create a cronjob

crontab -u www-data -e

and add the following line:

* * * * * /usr/bin/php /var/www/myMagentoSite.com/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/myMagentoSite.com/var/log/magento.cron.log

Finally, set the correct permissions:

sudo chown -R www-data: /var/www/myMagentoSite.com

Install and configure Nginx

Install Nginx from the official Ubuntu repositories::

sudo apt-get -y install nginx

Create a new Nginx server block with the following content:

sudo nano /etc/nginx/sites-available/myMagentoSite.com
upstream fastcgi_backend {
  server   unix:/run/php/php7.0-fpm.sock;
}

server {
    server_name myMagentoSite.com www.myMagentoSite.com;
    listen 80;
    set $MAGE_ROOT /var/www/myMagentoSite.com;
    set $MAGE_MODE developer; # or production

    access_log /var/log/nginx/myMagentoSite.com-access.log;
    error_log /var/log/nginx/myMagentoSite.com-error.log;

    include /var/www/myMagentoSite.com/nginx.conf.sample;        
}

Activate the server block by creating a symbolic link :

sudo ln -s /etc/nginx/sites-available/myMagentoSite.com /etc/nginx/sites-enabled/myMagentoSite.com

Delete the default configuration:

sudo rm -f /etc/nginx/sites-enabled/default

Test the Nginx configuration and restart nginx:

sudo nginx -t
sudo service nginx restart

You should be now able to login to your Magento back-end by going to http://myMagentoSite.com/admin_mejj1n using the information you set when running the bin/magento setup:install .

Install and configure Varnish

Installing Varnish is as simple as running the following command:

sudo apt-get install varnish

From you Magento Admin dashboard click on the STORES link (left sidebar) -> Configuration -> ADVANCED -> System -> Full Page Cache
Unselected Use system value and from the Caching Application list, select Varnish Cache (Recommended), save the configuration, click on the Varnish Configuration link and click on the Export VCL for Varnish 4 button. The varnish.vcl file which we will use will be exported in the /var/www/myMagentoSite.com/var/ directory.

Flush the Magento cache with:

sudo php bin/magento cache:flush

Delete the /etc/varnish/default.vcl and symlink it to the exported varnish configuration.

sudo rm -f /etc/varnish/default.vcl
sudo ln -sf /var/www/myMagentoSite.com/var/varnish.vcl /etc/varnish/default.vcl

To change varnish port from 6081 to 80, we need to edit the systemd service configuration.

Create a new customexec.conf file

sudo mkdir -p /etc/systemd/system/varnish.service.d
sudo nano /etc/systemd/system/varnish.service.d/customexec.conf

paste the following:

[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

and reload systemd units

sudo systemctl daemon-reload

Now we need to change Nginx listening port from 80 to 8080 and enable Nginx SSL termination with HTTP2, to do that open the Nginx configuration file and change it as follows:

sudo nano /etc/nginx/sites-available/myMagentoSite.com
upstream fastcgi_backend {
  server   unix:/run/php/php7.0-fpm.sock;
}

server {
    server_name myMagentoSite.com www.myMagentoSite.com;
    listen 8080;
    set $MAGE_ROOT /var/www/myMagentoSite.com;
    set $MAGE_MODE production; # or developer

    access_log /var/log/nginx/myMagentoSite.com-access.log;
    error_log /var/log/nginx/myMagentoSite.com-error.log;

    include /var/www/myMagentoSite.com/nginx.conf.sample;        
}

server {

    listen 443 ssl http2;
    server_name myMagentoSite.com www.myMagentoSite.com;

    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem; # change with your SSL cert
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key; # change with your SSL key
    ssl_protocols              TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers               'AES128+EECDH:AES128+EDH:!aNULL';
    ssl_session_cache    shared:SSL:10m;
    ssl_session_timeout 24h;
    keepalive_timeout 300s;

    location / {
        proxy_pass http://127.0.0.1;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Ssl-Offloaded "1";
        proxy_set_header      X-Forwarded-Proto https;
        proxy_set_header      X-Forwarded-Port 443;
        #proxy_hide_header X-Varnish;
        #proxy_hide_header Via;
        proxy_set_header X-Forwarded-Proto $scheme;

    }

}

If you don’t already have an SSL certificate, you can purchase a trusted SSL certificate here.

Restart Varnish and Nginx:

sudo systemctl restart nginx
sudo systemctl restart varnish

Change the base url to https and flush the cache

sudo bin/magento setup:store-config:set --base-url="https://myMagentoSite.com"
sudo php bin/magento cache:flush

If everything is setup correctly now you should be able to login to your Magento back-end by going to https://myMagentoSite.com/admin_mejj1n.

Stuck somewhere? Get a VPS from us and we’ll do all of this for you, free of charge!

Install and configure Redis caching

Redis is a key-value in memory data store and we will use it to replace the default Magento 2 Zend_Cache_Backend_File backend cache.  Install Redis by running the following command:

apt-get install php-redis redis-server

To configure your Magento installation to use Redis for session storage open the app/etc/env.php file and change/add the following:

sudo nano /var/www/myMagentoSite.com/app/etc/env.php

change:

  'session' =>
  array (
    'save' => 'files',
  ),

with:

'session' => 
   array (
   'save' => 'redis',
   'redis' => 
      array (
	'host' => '127.0.0.1',
	'port' => '6379',
	'password' => '',
	'timeout' => '2.5',
	'persistent_identifier' => '',
	'database' => '0',
	'compression_threshold' => '2048',
	'compression_library' => 'gzip',
	'log_level' => '1',
	'max_concurrency' => '6',
	'break_after_frontend' => '5',
	'break_after_adminhtml' => '30',
	'first_lifetime' => '600',
	'bot_first_lifetime' => '60',
	'bot_lifetime' => '7200',
	'disable_locking' => '0',
	'min_lifetime' => '60',
	'max_lifetime' => '2592000'
    )
),

and to use Redis for page caching add:

'cache' =>
array(
   'frontend' =>
   array(
      'default' =>
      array(
         'backend' => 'Cm_Cache_Backend_Redis',
         'backend_options' =>
         array(
            'server' => '127.0.0.1',
            'port' => '6379'
            ),
    ),
    'page_cache' =>
    array(
      'backend' => 'Cm_Cache_Backend_Redis',
      'backend_options' =>
       array(
         'server' => '127.0.0.1',
         'port' => '6379',
         'database' => '1',
         'compress_data' => '0'
       )
    )
  )
),

Finally flush the cache again:

sudo php bin/magento cache:flush

Further Optimizations

To further optimize your Magento installation from you Magento admin dashboard:

1. Go to STORES -> Configuration -> CATALOG -> Catalog -> Use Flat Catalog Category, select Yes and click Save Config.
2. Go to STORES -> Configuration -> ADVANCED -> Developer -> JavaScript Settings and set both Merge JavaScript Files and Minify JavaScript Files to Yes and click Save Config..
3. Go to STORES -> Configuration -> ADVANCED -> Developer -> CSS Settings and set both Merge CSS Files and Minify CSS Files to Yes and click Save Config.
4. Consider using a CDN – Content Delivery Network

Do not forget to flush the cache:

sudo php bin/magento cache:flush

All Credit for this HOW TO GOTO www.rosehosting.com Excellent Article

Magento 2.x: Required parameter ‘theme_dir’ was not passed — January 15, 2017

Magento 2.x: Required parameter ‘theme_dir’ was not passed

This error message appears in Magento 2 when you deleted a theme directory and the theme registry reminds in the database.

To fix this issue you need to go to app/design/frontend and check which themes you have installed.
Then go to the database and execute the following command:

SELECT * FROM theme;

Check what is the extra theme in that table and remove the registry.

Clear cache and that should be it.

Displaying Files like phpinfo.php Nginx — January 11, 2017

Displaying Files like phpinfo.php Nginx

Problem I was having was is basically I cannot run a simple info.php inside my public_html after doing some research I came up with this rewrite if you’ve any other solution do let me know it works for me.

Nginx

This is to display all files inside your /home/domain/public_html

location / {

location ~ ^/(.+.php)$ {
try_files $uri =404;
root $MAGE_ROOT;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include /etc/nginx/fastcgi_params;
}
try_files $uri $uri/ /index.php?$args;
}

 

Magento2 Set Anchor to YES in all Categories —

Magento2 Set Anchor to YES in all Categories

I am assuming you’ve PHPMYADMIN , not going in detail as to how you install that and all that crap

First identify the attribute id of the is_anchor attribute:

SELECT * FROM eav_attribute where attribute_code = 'is_anchor';

I get attribute 54 as my query not sure what you will get as that’s all depends on your environment

Now let’s run a Query to update all category to Anchor YES

UPDATE catalog_category_entity_int set value = 1 where attribute_id = 54;

TADA !!!!!!

Magento Page Cache Warming — January 8, 2017

Magento Page Cache Warming

Lets use a simple software to warm pages so they load faster and already in cache.

Before we do anything we need to install the following software

sudo apt-get install libxml-xpath-perl siege -y

Now lets setup Sitemap in Magento prior doing anything  – assuming that’s all taken care follow the following steps.

cat /home/yoursite/public_html/sitemap.xml | xpath -q -e "/urlset/url/loc/text()" > /root/tmp.urls

If the file is not local simply run the following

curl --silent http://example.com/sitemap.xml | xpath -q -e "/urlset/url/loc/text()" > tmp.urls

Now let’s make it warm

siege -v -c 1 -r `cat tmp.urls | wc -l` -f /root/tmp.urls

That’s it.

Magento 2 – (2.13) Failed to load resource: the server responded with a status of 404 (Not Found) **Solved —

Magento 2 – (2.13) Failed to load resource: the server responded with a status of 404 (Not Found) **Solved

After spending sometime with Magento 2 (2.13) , as I am running Nginx and PHP 7 fpm I was getting a lot of error and after some research I was able to find a solution simply replace location static in given magento 2 nginx configuration file and restart nginx and php fpm server to reflect changes.

location /static/ {
    if ($MAGE_MODE = "production") {
      expires max;
    }

    # Remove signature of the static files that is used to overcome the browser cache
    location ~ ^/static/version {
      rewrite ^/static/(versiond*/)?(.*)$ /static/$2 last;
    }

    location ~* .(ico|jpg|jpeg|png|gif|svg|js|css|swf|eot|ttf|otf|woff|woff2)$ {
      add_header Cache-Control "public";
      add_header X-Frame-Options "SAMEORIGIN";
      expires +1y;

      if (!-f $request_filename) {
        rewrite ^/static/(versiond*/)?(.*)$ /static.php?resource=$2 last;
      }
    }

    location ~* .(zip|gz|gzip|bz2|csv|xml)$ {
      add_header Cache-Control "no-store";
      add_header X-Frame-Options "SAMEORIGIN";
      expires off;

      if (!-f $request_filename) {
         rewrite ^/static/(versiond*/)?(.*)$ /static.php?resource=$2 last;
      }
    }

    if (!-f $request_filename) {
      rewrite ^/static/(versiond*/)?(.*)$ /static.php?resource=$2 last;
    }

    add_header X-Frame-Options "SAMEORIGIN";
  }

 

How to install Nginx Google Page Speed on Ubuntu 16.10 x64 Xenial — January 4, 2017

How to install Nginx Google Page Speed on Ubuntu 16.10 x64 Xenial

Happy New Year !!!!!!

I am not going to bore you guys, I’ve already completed the building process and the hard work so you can enjoy .

Install Nginx with Google Page speed

At the time of building this we’ve latest stable version of

nginx version: nginx/1.10.2
built by gcc 6.2.0 20161005 (Ubuntu 6.2.0-5ubuntu12)
built with OpenSSL 1.0.2g 1 Mar 2016
TLS SNI support enabled

You need to install nginx deb package by simply running the following command

nginx_1.10.2-1-xenial_amd64.deb

Download Nginx Google Pagespeed

How to install Nginx Page Speed on Debian x64 Jessie —

How to install Nginx Page Speed on Debian x64 Jessie

Happy New Year !!!!!!

I am not going to bore you guys, I’ve already completed the building process and the hard work so you can enjoy .

Install Nginx with Google Page speed

At the time of building this we’ve latest stable version of

nginx version: nginx/1.10.2

built by gcc 4.9.2 (Debian 4.9.2-10)
built with OpenSSL 1.0.1t 3 May 2016
TLS SNI support enabled

You need to install nginx deb package by simply running the following command

dpkg -i nginx_1.10.2-1-jessie_amd64.deb

Download Nginx Google Pagespeed

Google Pagespeed latest version as of Jan-4-2017