Trends in Mendix Cloud v4

Last update: Edit

1 Introduction

To track the usage growth of your app, and to debug performance problems, the Mendix Cloud includes detailed graphs of both your app and its environment. These graphs show performance trends of your apps in the paid editions of the Mendix Platform. If you experience issues with your app, always check the Alerts and Trends in the Developer Portal under Operate.

To view the Trends, you must have permission to Access the Monitoring.

Assign permission by following these steps:

  1. Open your app in the Developer Portal.
  2. Click Security under the Settings category on the left.
  3. Go to the Node Permissions tab.
  4. Choose the environment for which you want to grant access.
  5. Check Access to Monitoring next to the name of the person to whom you want to grant this permission.

You can find the trends by following these steps:

  1. Open your app in the Developer Portal.
  2. Click Metrics under the Operate category.
  3. Select the environment you want to monitor under the tab Trends.

3.2 Interpreting the Graphs

As with all complex IT systems, there are many interrelated components which can cause performance issues. This document cannot cover all possibilities, but is intended as a general introduction to the information which is displayed and a few ideas about where to look for possible areas of concern.

3.2.1 Disk Partitions

If there is more than one disk partition in the system, the /srv partition generally contains project files and uploaded files of the application, while /var generally holds the database storage.

3.2.2 Combining Information

You can often get more information about the performance of your app by combining the information from several graphs. Useful graphs for this are:

For example, a combination of a moderate number of IO operations, low disk throughput, high CPU usage, full memory disk cache, and reports of long running database queries in the application log could point to a shortage of system memory for disk cache that leads to repeated random reads from disk storage.

4 Application Statistics

This section explains the metrics that represent the current status and statistics of a running Mendix application. This includes requests that the application processes from the services/clients with which it is integrated as well as Java Virtual Machine-related statistics and the Jetty Web server it uses.

The Number of handled external requests graph shows the number of requests that are sent from the client and systems that integrate with your application using web services.

The number of requests per second is split up by request handlers. The key ones are:

  • xas/ lists general queries for data in data grids, sending changes to the server, and triggering the execution of microflows
  • ws/ shows the number of web service calls that were done
  • file shows the number of file uploads and downloads
  • The default should not list any requests, because static content is directly served to the user by the front-facing web server, which is placed between the user and this application process

These are the commonly used types:

Request Type Explanation
api-doc/ A general API doc overview for other doc request handlers.
debugger/ Mendix Runtime debugger request handler.
default Default request handler.
file Request handler used for serving files.
odata-doc/ Documentation request handler for OData.
openid/ OPENID authentication request handler.
p/ Request handler for custom page URLs.
rest-doc/ HTTP REST web service request handler documentation.
ws-doc/ SOAP webservice request handler documentation.
ws/ SOAP webservice call request handler.
xas/ Request handler used by the Mendix Runtime itself.

Additional information about request handlers is available in the Requests section of Monitoring Mendix Runtime and the Applying Access Restrictions to Unnecessary Request Handlers section of How To Implement Best Practices for App Security.

In the Object cache graph you can monitor the number of Mendix Objects that live in memory.

Non-persistable entities live in the JVM memory and are garbage-collected regularly. If you have a memory leak, the number of objects in memory will grow over time. This might be a problem.

The User Accounts and Login Sessions graph shows the number of logged-in named and anonymous user sessions for your application.

These are the user types:

User Type Explanation
named users Total number of user accounts.
concurrent named user sessions Total number of sessions for users using a named login.
concurrent anonymous user sessions Total number of sessions for users who are logging in anonymously.

The JVM Object Heap graph shows the internal distribution of allocated memory inside the application process for Java objects. Java objects are created in Java actions, but also include all objects that are used by microflows running in your app at runtime.

One of the most important things to know, in order to be able to interpret the values in this graph, is that the JVM does not immediately clean up objects that are no longer in use. This graph will show unused memory as still in use until the so-called garbage collector, which analyzes the memory to free up space, is run. So, you cannot see how much of the JVM memory that is in use before a garbage collection will be available after the garbage collection cycle, because the garbage collection process will only find that out when it actually runs.

These are the types of object:

Object Type Explanation
tenured generation As objects “survive” repeated garbage collections in the eden space, they are migrated to the tenured generation. You can look at this metric as a number of long-living objects in JVM.
native memory The native memory is the memory available to the operating system.
eden space The pool from which memory is initially allocated for most objects.
unused Unused JVM heap memory.

For example, if the tenured generation is shown as 65% of the complete heap size, this may change to 0% if a garbage collection is triggered when the percentage reaches two thirds of the total heap size. However, it could stay at this 65% if all data in this memory part is still referenced by running actions in the application. This behavior means that the JVM heap memory graphs are the most difficult to base conclusions on.

This JVM Process Memory Usage graph is similar to the previous graph, JVM Object Heap. It shows a more complete view of the actual size and composition of the operating system memory that is in use by the JVM process.

This graph is primarily present to provide more insight in situations where the part of the real used memory outside the JVM Object Heap is growing too much, causing problems with memory shortage in the operating system.

More information on this graph is available in a Tech Blog post: What’s in my JVM memory?.

These are the types:

Type Explanation
code cache JVM includes a code cache, containing memory that is used for compilation and storage of native code.
native code JVM allocates a certain amount of memory space for native bytecode.
jar files JAR files necessary for JVM itself to run.
tenured generation As objects “survive” repeated garbage collections in the survivor space, they are migrated to the tenured generation. You can look at this metric as a number of long living objects in JVM.
survivor space The pool containing objects that have survived the garbage collection of the Eden space.
eden space The pool from which memory is initially allocated for most objects.
unused java heap Unused JVM heap memory.
permanent generation The pool containing all the reflective data of the virtual machine itself, such as class and method objects. With Java VMs that use class data sharing, this generation is divided into read-only and read-write areas.
other Virtual or reserved memory space.
thread stacks Stacks that are reserved for unique threads.

The Application node operating system memory graph shows the distribution of operating system memory that is available for this server.

Performance issues can arise if the apps memory takes up too large a proportion of the operating system memory.

The Threadpool for handling external requests graph shows the number of concurrent requests that are being handled by the Mendix Runtime. The requests are counted in two circumstances:

  • they are initiated by a remote API – the way the normal web-based client communicates
  • they are initiated by calling web services

Because creating a new thread that can concurrently process a request is an expensive operation, Mendix holds a pool of threads that can quickly start processing new incoming requests. This pool automatically grows and shrinks according to the number of requests that are flowing through the application.

The values shown by the graph are:

Value Explanation
min threads Minimum bound of threads to be used by the Jetty threadpool.
max threads Maximum bound of threads to be used by the Jetty threadpool.
active threads Active threads that are being used within the Jetty threadpool.
threadpool size The current total size of the Jetty threadpool.

The Total Number of Threads in the JVM Process graph shows the total number of threads that exist inside the running JVM process.

Besides the threadpool that is used for external HTTP requests, described above, this includes the threadpool used for database connections, internal processes inside the Mendix Runtime, and optional extra threads created by the application itself, for example, using a threadpool in a custom module or custom Java code.

The Application node CPU usage graph shows the CPU utilization in percentage.

The Application node disk usage (percentage) graph shows the relative amounts of data that are stored on disk.

This graph should be interpreted in combination with other graphs. See Combining Information, above.

The disk usage graph shows only the disk usage inside the container. This is usually only relevant if your application creates a lot of temporary files in /tmp. This value is not the same as the file document storage.

5 Database Statistics

In this section you will find the statistics about the database that the application uses.

The Number of database queries being executed graph shows the number of database queries that are executed by your Mendix application.

The queries are broken down into queries that actually modify data (insert, update, and delete) and queries that fetch data (select).

These are the types of query:

Type Explanation
inserts Number of SQL INSERT INTO statements per second. These add new rows of data to a table in the database.
transactions Number of SQL transactions per second. A transaction is a unit of work that is performed against a database.
update Number of SQL UPDATE statements per second. The SQL UPDATE query modifies the existing records in a table.
select Number of SQL SELECT statements per second. The SQL SELECT statement fetches data from a database table that returns this data in the form of a result table.
delete Number of SQL DELETE statements per second. The SQL DELETE query deletes the existing records from a table.

The Database table vs. index size graph shows the distribution between disk space used for storing indexes and actual data.

Remember, indexes actually occupy memory space and disk storage, as they are just copies of parts of your data stored and sorted in another way! Besides the data you are processing, the relevant parts of the indexes also have to be read into system memory to be able to use them.

These are the values:

Value Explanation
tables Total space taken by the database.
indices Amount of space taken by the indices in the database.

The Number of database tuple mutations graph shows the number of database objects that were actually changed by database queries from the application.

For a single database operation that affects more than one object, this graph shows the number of objects actually changed, as measured from inside the database. However, the Number of database queries being executed graph will only show a single database query for the same operation.

These are the values:

Value Explanation
xact commit Number of transactions committed per second.
xact rollback Number of transactions rolled back per second.
xact inserted Number of tuples inserted per second.
xact updated Number of tuples updated per second.
xact deleted Number of tuples deleted per second.

The Number of database connections graph shows the number of connections to the PostgreSQL server.

This should go up and down with the usage of the application. The number of connections is limited to 50.

The Database node operating system memory graph shows the distribution of operating system memory that is available for this server.

It is crucial to the performance of an application that parts of the database data and indexes that are referenced a lot are always available in the working memory of the server. A lack of Freeable memory on a busy application will result in continuous re-reads of data from disk, which takes several orders of magnitude more time, slowing down the entire application. This may indicate that you have a large number of concurrent database connections from your app and that the environment is not large enough to support these.

These are the types:

Type Explanation
Used memory Total memory size of the database instance minus the freeable memory.
Freeable memory Memory that is allocated dynamically containing, for example, cached reads and indexes.
Swap usage The amount of swap space used on the database instance.

The Database node CPU usage graph shows the amount of CPU usage over time, as a percentage.

The Database node disk throughput graph shows the amount of data that is being read from and written to disk.

If you see large values here which do not immediately drop back again, it may indicate that your app is continually swapping data to disk. This could be caused by inefficient queries, for example ones which require sorting within the app.

The Database node disk usage (in bytes) graph displays the absolute amount of data that is stored on disk.

The Database node disk IO/s graph shows the number of disk read and write operations that are done from and to the disk storage. It does not show the amount of data that was transferred.

There are two sets of values:

Value Explanation
read Read ops on the disk holding the database.
write Write ops on the disk holding the database.

The Database node disk latency graph shows the average waiting times for disk operations to complete.

Interpreting the values in this graph should be done in combination with the other disk stats graphs, together with the type of requests that were made. Sequential or random reads and writes can create a different burden for disk storage.

There are two sets of values:

Value Explanation
read Read ops on the disk holding the database.
write Write ops on the disk holding the database.

6 Read More