The Definitive Guide on Magento Functional Testing Framework (MFTF)

The eCommerce & retail ecosystem has become so fiercely competitive for Magento store owners to survive. It has become vital for them to continuously monitor the market and adapt to it. This allows companies to quickly respond to an ever-changing retail sphere. While responding, store owners also need to take note of and ensure the user experience on their online storefront is smooth for all visitors.

In wake of the Magento coding issues and bugs, Magento released the automated testing framework called MFTF (Magento Functional Testing Framework). And this guide perfectly dives deep into what MFTF is and how store owners can use it to find root causes to solve the bottlenecks and accomplish their goal of running the most functional Magento website. Let’s first get to know what is MFTF.

What is Magento Functional Testing Framework?

MFTF is an open-source testing framework developed & released by the Magento team for its Magento ecosystem. It is a framework that helps developers and Magento website owners to perform automated end-to-end functional tests.

MFTF is an evolution of the Magento Testing Framework (MTF). The framework performs various readymade tests to check your Magento 2 website’s functionalities are working as expected or not. You can perform these tests any time with the options of:

• Running an independent & single test only.
• Running multiple tests simultaneously.
• Running all available tests at once.

The goal here is to find any possible discrepancies between the expected & real behavior of your Magento websites’ functionality.

Why Functional Testing is Needed in Magento?

Magento 2 comes with a hard to grasp & complex architecture, which makes it difficult to test every functionality manually. And due to the complex flow of Magento, manual testing is subject to human error. Although there are unit testing & integration testing methods that can cover almost the whole Magento code. Still, it does not guarantee that everything will be in order & execute the same way as on the other devices & browsers.

The automated MFTF testing framework spares the lives of QA specialists to script & enable those scripts to write test cases. Furthermore, the tool provides a set of functions that help use the existing test as a viable template to write new ones too.

However, it requires the correct configuration to use the whole range of functions offered by the MFTF tool. Configuring the MFTF is usually part of Magento Code Audit Service or Magento Security Audit Service.

The best thing about MFTF is it only uses XML for test case writing and no PHP is involved in this. But at the later stage of the testing, XML test cases are auto-converted into PHP scripts & then the tool tests it by employing Codeception, Selenium, or Allure.

Advantages of Magento Functional Testing Framework

Here’re some reasons (advantages) to apply MFTF for your Magento project:

• Thanks to the modularity support, it offers flexibility.
• It offers customization for existing tests.
• The intuitive reporting tool makes it an informative testing framework.
• Because of the range of web driver selection options, the framework is compatible.
• The framework helps reduce the testing time significantly.
• The framework is suitable for different Magento editions & versions & performs tests from scratch.
• It can really come in handy for the long-term Magento development project.
• Also, it helps detect version-specific bugs by testing vigorously.
• The framework comes out-of-the-box ready to use tests.
• And the best thing about the MFTF is its scalability for a project.

Why Use Magento Functional Testing Framework to Automate Testing?

Thanks to the low-entry threshold, the MFTF is an easy-to-start & learn automation tool. Moreover, the tool is best suited for test automation, and here’s why,

• The tool is useful in improving the efficiency of a development team by significantly reducing test times.
• MFTF helps perform regression testing in a way to minimize the effort & facilitate functional testing.
• The manual testing that took over 7 hours before, takes only 2 hours with MFTF automated tests. The whopping 5 hours saving can be utilized on other things and enhance the productivity of a team.
• Regardless of your budget for implementing the automated tests on your Magento website, it can help improve the quality of your work & the coding of your website.
• After you have a thorough understanding of the framework, you can add and deploy new test cases for new & custom functionalities.
• MFTF acts as a dedicated Magento solution & enables you to learn new solutions for Magento too.

Things to Know, Have & Learn Before You Start Applying MFTF

• There is no requisite for you to know any programming language. The basic knowledge of XML tags, & Xpath is enough for you to find any element on the site.
• Your Magento website should at least have a Magento 2.2.X or higher version.
• The website server must run on the supported PHP version.
• Use Selenium tool’s server standalone 3.1 or the higher.
• The version of Java 1.8 or later must have been installed.
• Must install the Composer 1.3 or later version.
• Install ChromeDriver 2.33 or later or any other supported webdriver in the same directory.

Getting Started with MFTF:

Using existing Composer dependency like magento/magento2-functional-testing-framework you can install the framework on your system. Once installed, you can start to build your project, edit the environmental settings, enable MFTF to send Magento CLI commands to your Magento, run tests with the help of Selenium server & MFTF commands, and generate reports in CLI. Also, the Allure framework allows you to generate visually representative data reports.

To create functional tests for your website, you need to have a max of six folders in MFTF. Let’s understand what those are,

• Test Folder: The test folder is where we will put all our test cases. They are nothing but simple XML files containing single or multiple tests. As per the convention, you should put at least one test in each file having the same file name as the test name and should follow the camel casing.
• Page Folder: Here, the defined pages will be visited while the test case execution is in the process. The naming convention is the same as the test folder with file name & page name being similar & must follow camel casing with a suffix “Page” at the end of the name. There can be multiple sections in a page and each section will have multiple numbers of elements such as div, form, anchor, button, checkbox, etc.
• Section Folder: Just like a header, footer, sidebar, content, etc., a section is a part of the page. You are free to divide the page into as many sections as you like. Creating only one section & defining all the elements there is not a feasible option as it will affect the readability of the page & becomes hard to understand too.
• Action Group Folder: It is a group of actions such as page load, click on the button, etc. An action group is useful in creating actions for better reusability of actions.
• Data Folder: For any test case to complete, it requires dummy data for a product, for the customer, & others to test product & login features. Here, all the XML files must have the suffix “Data” at the end.
• Metadata Folder: Sometimes, you may need to create some data entities at runtime to use while executing test cases. This is made possible with the metadata folder.

Best Practices with Magento Functional Testing Framework:

The fundamental principle while developing tests is,

“To wrap a set of actions, use action groups so that you can reuse them multiple times. When a test or action group needs to be repeated except a few steps, use an extension.”

Other than this here’re some of the best practices you must keep in mind with MFTF:

• Independence means the test result of one doesn’t affect the test result of others. Even though multiple tests are performed at once(for instance, multiple tests on different browsers) they should not interfere with each other.
• After a test has been executed, it should delete all the critical data so that it does not interfere with the subsequent execution of other tests or the same test. (Deletion may not be always mandatory, but the field may set to be optional that was mandatory for the test)
• When it comes to IDs, Keys, URLs, names, & others, all the data should be unique for the same purpose created by Magento tests.
• Do not neglect the Standard Magento Action Groups.
• Autotests must be self-sufficient & should not depend on the data that may change or disappear at any time.
• If your Magento does not have any bugs, then the tests must be completed. This seems obvious at first but is not easy to achieve.

MFTF Lifehacks:

1) A very useful way to maintain the order in tests & running them at a later stage is by committing to the same value options. This may seem an unsophisticated way of doing so. Here’s how you can differentiate all the possible values and use them in each test,

• A common group of values for all tests developed within the company to run everything conveniently.
• A group with the same name as your test subject consisting of all the tests for that test subject.
• Another group can be subdivided that corresponds to the Magento functionality e.g., tests affecting checkout are combined into checkout group.

2) With several tests having the same preconditions, don’t neglect using test suites. It reduces the time required to run tests significantly.

3) It is a great practice to run tests automatically once the master branch updates.

4) Once the tests are performed, it’s good to keep the test statistics so you can access them in the future to know how they detect bugs. This can be helpful in cases where tests are running several times without detecting any bugs. This may also signal you to analyze the test scenario & improve them too.

5) While planning resources and time allocation, don’t forget to allocate time for test maintenance & fixes. All tests including MFTF tests require these activities to be performed at a certain interval.

6) Lastly, don’t forget to generate & use reports for added benefits.

Final Words

Magento Functional Testing Framework is astonishingly helpful in automating regression testing. It helps you perform almost all the functionality tests automatically without much manual effort. However, doing so can be tricky for many store owners coming from a non-technical background. Moreover, understanding the test reports is not that easy job too.

That’s when the experts at M-Connect Media can come to your rescue. We offer flamboyant Magento services including theme development, migration, upgrade, PSD conversion, page speed optimization. Magento Functional Testing comes as part of our immaculate Magento code audit service.

Contact us now for further discussion on the same.