
Leading the Way: Payara Platform Community 7 Beta Now Fully Jakarta EE 11 Certified
We’re excited to announce that Payara Platform Community 7 Beta application server is now fully certified as Jakarta EE 11 […]
Our Payara Engineers have been working very hard on lots of new features ready for our final 5.181 release! One of the key features we intend to deliver is compatibility with MicroProfile 1.2, which will include (among other things) a Fault Tolerance API.
The Eclipse MicroProfile project is a collaboration between application server vendors and the community. The initial aim was to create a project that would allow developers with skill in Java EE to use that knowledge in creating modern cloud-native or microservice-based applications.
Along with the benefits of modularising and distributing a system across multiple instances, there also come new challenges. In a monolith, the whole system is either up or down. With microservices, part of the system can experience any number of problems which can propagate through the rest of the system in very unpredictable and subtle ways.
To mitigate this, there are a number of design patterns which have emerged to make microservice architectures more fault tolerant. These patterns have all been incorporated into the MicroProfile Fault Tolerance specification and include:
Source: MicroProfile Fault Tolerance 1.0 Architecture
To demonstrate some of the fault tolerance behaviour available in Payara Micro (and Payara Server), I have created a simple example which demonstrates Retry, Fallback and Timeout which can be found in the payara/payara-examples project on GitHub, in the microprofile/fault-tolerance directory.
There are 2 examples of fault tolerance. One which demonstrates the @Retry
annotation to re-attempt an execution which has failed, and another which demonstrates a @Timeout
with a @Fallback
method to handle a slow invocation.
mvn clean install
To run the Uber JAR, runjava -jar target/fault-tolerance-1.0-SNAPSHOT-microbundle.jar
To run the WAR file on a different Payara Micro instance, runjava -jar /path/to/payara-micro.jar -deploy target/fault-tolerance-1.0-SNAPSHOT.war
Watch the output of Payara Micro to see the URLs created, and visit the endpoints to trigger an example of fault tolerance. Watch the logs to see how Payara Micro behaves.
The example is a simple JAX-RS class with two @GET
methods, getEmployeeById
and getAllEmployees
. There is a list of 4 people which can be returned.
There are 2 methods to introduce simple problematic behaviour, one isDown()
which simply returns a boolean based on Math.random()
and isSlow()
which will sleep for a second based on Math.random()
@Retry
annotation has been added to the getEmployeeById
, where the isDown()
check may cause a RuntimeException
.@GET
@Path("{id}")
@Retry(maxRetries = 4, retryOn = {RuntimeException.class})
public String getEmployeeById(@PathParam("id") int id) {
System.out.println("Called getEmployeeById a total of " + ++retryCounter + " times");
if (id >= employees.size()) return "No such employee. Try a number lower than " + employees.size();
if (isDown()) throw new RuntimeException();
return employees.get(id);
}
private boolean isDown() {
// approx 80% chance
return Math.random() > 0.2;
}
There is a retryCounter
to show the total amount of times the method has been called, so we can see the retry in action. The isDown()
method should cause failures around 80% of the time but, with 4 retries, this probability is reduced. If the Retry eventually gets a success, then the result will be returned and the user will not see any problem. If the Retry gets a failure every time, then it will fail as normal and the user will see an ungraceful exception.
An example of the log output for one test is below. I invoked the method 3 times, and have indicated this among the log messages:
---> invoke <---
Called getEmployeeById a total of 1 times
---> invoke <---
Called getEmployeeById a total of 2 times
[2018-01-10T09:25:19.063+0000] [] [INFO] [] [fish.payara.microprofile.faulttolerance.interceptors.RetryInterceptor] [tid: _ThreadID=20 _ThreadName=http-thread-pool::http-listener(1)] [timeMillis: 1515576319063] [levelValue: 800] Retrying as long as maxDuration isnt breached, and no more than {0} times
Called getEmployeeById a total of 3 times
Called getEmployeeById a total of 4 times
Called getEmployeeById a total of 5 times
---> invoke <---
Called getEmployeeById a total of 6 times
[2018-01-10T09:25:37.861+0000] [] [INFO] [] [fish.payara.microprofile.faulttolerance.interceptors.RetryInterceptor] [tid: _ThreadID=25 _ThreadName=http-thread-pool::http-listener(6)] [timeMillis: 1515576337861] [levelValue: 800] Retrying as long as maxDuration isnt breached, and no more than {0} times
Called getEmployeeById a total of 7 times
Called getEmployeeById a total of 8 times
Called getEmployeeById a total of 9 times
Called getEmployeeById a total of 10 times
[2018-01-10T09:25:38.147+0000] [] [WARNING] [] [javax.enterprise.web] [tid: _ThreadID=25 _ThreadName=http-thread-pool::http-listener(6)] [timeMillis: 1515576338147] [levelValue: 900] [[
StandardWrapperValve[RestApplication]: Servlet.service() for servlet RestApplication threw exception
java.lang.RuntimeException
at EmployeeResource.getEmployeeById(EmployeeResource.java:36)
at org.jboss.weld.proxies.EmployeeResource$Proxy$_$_WeldSubclass.getEmployeeById$super(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51)
In this example, the first invocation worked. The second did not work, and the Retry began to work, as shown in the log message. The method is then tried 4 more times and, on the 3rd attempt (number 5 shown by the counter), the method returned successfully. Next, I tried the invocation again and found that the method did not succeed, even after 4 retries, so an exception was thrown.
Both the @Timeout
and @Fallback
annotations have been added to getAllEmployees()
, and an extra method added called getAllEmployeesFallback()
to handle invocations when the @Timeout
is triggered.
private final long TIMEOUT = 500;
private final long SLEEPTIME = 1000;
@GET
@Fallback(fallbackMethod = "getAllEmployeesFallback")
@Timeout(TIMEOUT)
public String getAllEmployees() throws InterruptedException {
if (isSlow()) return employees.toString();
return employees.toString();
}
public String getAllEmployeesFallback() {
return "It took longer than expected to get all employees. Try again later!";
}
private boolean isSlow() throws InterruptedException {
if (Math.random() > 0.4) {
// approx 60% chance
Thread.sleep(SLEEPTIME);
return true;
}
return false;
}
In this example, the method will sporadically sleep for longer than the configured timeout. In this case, the @Fallback
will intercept the invocation and call getAllEmployeesFallback()
to give the user a different message.
There are minimal configuration options for Fault Tolerance through asadmin commands. The only configurable aspect relates to the @Asynchronous
annotation, which will ensure that the execution of the client request will be on a separate thread. There are two asadmin commands which allow you to get and set the ManagedExecutorService name or ManagedScheduledExecutorService name as follows:
Usage: asadmin [asadmin-utility-options] set-fault-tolerance-configuration
[--managedexecutorservicename <managedexecutorservicename>]
[--managedscheduledexecutorservicename <managedscheduledexecutorservicename>]
[--target <target(default:server-config)>]
[-?|--help[=<help(default:false)>]]
Fault Tolerance is just one of several MicroProfile 1.2 specs that are already implemented, with more on the way.
Stay tuned for more updates on using MicroProfile in Payara Server/Micro!
We’re excited to announce that Payara Platform Community 7 Beta application server is now fully certified as Jakarta EE 11 […]
Welcome aboard the August 2025 issue of The Payara Monthly Catch! With summer in full swing, things may have felt […]
Enterprise Java applications power global commerce, healthcare, government and countless other industries. These systems must be scalable, secure and […]
When I try this example with @Fallback(fallbackMethod = “microhustleFallback”) I am getting the following exception. And also I have defined my method in the same class
[2019-05-18T12:09:38.540+0530] [] [[1;91mSEVERE[0m] [NCLS-CORE-00026] [[1;94mjavax.enterprise.system.core[0m] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1558161578540] [levelValue: 1000] [[
Exception during lifecycle processing
org.glassfish.deployment.common.DeploymentException: CDI definition failure:Could not find fallback
method: microhustleFallback — Could not find fallback method: microhustleFallback
at org.jboss.weld.bootstrap.events.ContainerLifecycleEvents.fireProcessAnnotatedType(ContainerLifecycleEvents.java:187)
at org.jboss.weld.bootstrap.events.ContainerLifecycleEvents.fireProcessAnnotatedType(ContainerLifecycleEvents.java:171)
at org.jboss.weld.bootstrap.BeanDeployer.processAnnotatedTypes(BeanDeployer.java:166)
at org.jboss.weld.bootstrap.BeanDeployment.createTypes(BeanDeployment.java:219)
at org.jboss.weld.bootstrap.WeldStartup.startInitialization(WeldStartup.java:415)
at org.jboss.weld.bootstrap.WeldBootstrap.startInitialization(WeldBootstrap.java:79)
at org.glassfish.weld.WeldDeployer.processApplicationLoaded(WeldDeployer.java:514)
at org.glassfish.weld.WeldDeployer.event(WeldDeployer.java:428)
at org.glassfish.kernel.event.EventsImpl.send(EventsImpl.java:131)
at org.glassfish.internal.data.ApplicationInfo.load(ApplicationInfo.java:333)
at com.sun.enterprise.v3.server.ApplicationLifecycle.prepare(ApplicationLifecycle.java:497)
at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:540)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:549)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:545)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:544)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:575)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:567)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:566)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1475)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1300(CommandRunnerImpl.java:111)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1857)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1733)
at com.sun.enterprise.admin.cli.embeddable.DeployerImpl.deploy(DeployerImpl.java:136)
at fish.payara.micro.impl.PayaraMicroImpl.deployAll(PayaraMicroImpl.java:1536)
at fish.payara.micro.impl.PayaraMicroImpl.bootStrap(PayaraMicroImpl.java:1028)
at fish.payara.micro.impl.PayaraMicroImpl.main(PayaraMicroImpl.java:200)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at fish.payara.micro.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at fish.payara.micro.boot.loader.Launcher.launch(Launcher.java:107)
at fish.payara.micro.boot.loader.Launcher.launch(Launcher.java:70)
at fish.payara.micro.boot.PayaraMicroLauncher.main(PayaraMicroLauncher.java:79)
at fish.payara.micro.PayaraMicro.main(PayaraMicro.java:397)
]]
[2019-05-18T12:09:38.542+0530] [] [[1;91mSEVERE[0m] [] [[1;94mjavax.enterprise.system.core[0m] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1558161578542] [levelValue: 1000] Exception while loading the app
Hi,
Can you share us the JAX-RS resource method and fallback method on the Payara Forum (https://groups.google.com/forum/#!forum/payara-forum) or the Github Issues page (https://github.com/payara/Payara/issues) for further investigation.
It seems a signature issue of your ‘microhustleFallback’ method.
Thanks
Rudy