For reprinting, please indicate the source: Translation: Hystrix - How To Use
Hello World!
The following code shows the HystrixCommand version of Hello World:
public class CommandHelloWorld extends HystrixCommand<String> { private final String name; public CommandHelloWorld(String name) { super(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")); this.name = name; } @Override protected String run() { // a real example would do work like a network call here return "Hello " + name + "!"; } }
The same implementation of HystrixObservableCommand is as follows:
public class CommandHelloWorld extends HystrixObservableCommand<String> { private final String name; public CommandHelloWorld(String name) { super(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")); this.name = name; } @Override protected Observable<String> construct() { return Observable.create(new Observable.OnSubscribe<String>() { @Override public void call(Subscriber<? super String> observer) { try { if (!observer.isUnsubscribed()) { // a real example would do work like a network call here observer.onNext("Hello"); observer.onNext(name + "!"); observer.onCompleted(); } } catch (Exception e) { observer.onError(e); } } } ).subscribeOn(Schedulers.io()); } }
Synchronous Execution
Synchronized execution can be achieved by calling the HystrixCommand.execute() method, as follows:
String s = new CommandHelloWorld("World").execute();
The tests are as follows:
@Test public void testSynchronous() { assertEquals("Hello World!", new CommandHelloWorld("World").execute()); assertEquals("Hello Bob!", new CommandHelloWorld("Bob").execute()); }
HystrixObservableCommand does not provide a synchronous execution method, but if it is determined that only one value will be generated, it can also be implemented as follows:
- HystrixObservableCommand.observe().observe().toBlocking().toFuture().get()
- HystrixObservableCommand.toObservable().observe().toBlocking().toFuture().get()
If multiple values are actually generated, the above code throws java.lang.IllegalArgumentException: Sequence contains too many elements.
Asynchronous Execution
Asynchronous execution can be achieved by calling the HystrixCommand.queue() method as follows:
Future<String> fs = new CommandHelloWorld("World").queue();
At this point, command execution results can be obtained through the Future.get() method:
String s = fs.get();
The test code is as follows:
@Test public void testAsynchronous1() throws Exception { assertEquals("Hello World!", new CommandHelloWorld("World").queue().get()); assertEquals("Hello Bob!", new CommandHelloWorld("Bob").queue().get()); } @Test public void testAsynchronous2() throws Exception { Future<String> fWorld = new CommandHelloWorld("World").queue(); Future<String> fBob = new CommandHelloWorld("Bob").queue(); assertEquals("Hello World!", fWorld.get()); assertEquals("Hello Bob!", fBob.get()); }
The following two implementations are equivalent:
String s1 = new CommandHelloWorld("World").execute(); String s2 = new CommandHelloWorld("World").queue().get();
HystrixObservableCommand does not provide a queue method, but if it is determined that only one value will be generated, it can also be implemented as follows:
- HystrixObservableCommand.observe().observe().toBlocking().toFuture()
- HystrixObservableCommand.toObservable().observe().toBlocking().toFuture()
If multiple values are actually generated, the above code throws java.lang.IllegalArgumentException: Sequence contains too many elements.
Reactive Execution
You can also use HystrixCommand as an Observable object to observe its results, using any of the following methods:
- observe(): Once this method is invoked, the request will be executed immediately. Using the ReplaySubject feature, it ensures that no command-generated results will be lost, even if the results are generated before you subscribe.
- toObservable(): When this method is called, the request is not executed immediately, but when a subscriber subscribes.
Observable<String> ho = new CommandHelloWorld("World").observe(); // or Observable<String> co = new CommandHelloWorld("World").toObservable();
Then you can subscribe to this Observable to get the command results:
ho.subscribe(new Action1<String>() { @Override public void call(String s) { // value emitted here } });
The tests are as follows:
@Test public void testObservable() throws Exception { Observable<String> fWorld = new CommandHelloWorld("World").observe(); Observable<String> fBob = new CommandHelloWorld("Bob").observe(); // blocking assertEquals("Hello World!", fWorld.toBlockingObservable().single()); assertEquals("Hello Bob!", fBob.toBlockingObservable().single()); // non-blocking // - this is a verbose anonymous inner-class approach and doesn't do assertions fWorld.subscribe(new Observer<String>() { @Override public void onCompleted() { // nothing needed here } @Override public void onError(Throwable e) { e.printStackTrace(); } @Override public void onNext(String v) { System.out.println("onNext: " + v); } }); // non-blocking // - also verbose anonymous inner-class // - ignore errors and onCompleted signal fBob.subscribe(new Action1<String>() { @Override public void call(String v) { System.out.println("onNext: " + v); } }); }
Using Java 8's Lambda expression can make the code more concise:
fWorld.subscribe((v) -> { System.out.println("onNext: " + v); }) // - or while also including error handling fWorld.subscribe((v) -> { System.out.println("onNext: " + v); }, (exception) -> { exception.printStackTrace(); })
Information about Observable can be See here
Reactive Commands
Instead of converting HystrixCommand into an Observable using the above method, you can also choose to create a Hystrix ObservableCommand object. The Observable wrapped in Hystrix ObservableCommand allows multiple results, whereas Hystrix Command can only produce one result even if converted into Observable.
When using Hystrix Observable Commnad, you need to overload the construct method to implement your business logic, not the run method, and the contruct method will return the Observable you need to wrap.
Observable objects can be obtained from Hystrix Observable Command using any of the following methods:
- observe(): Once this method is invoked, the request will be executed immediately. Using the ReplaySubject feature, it ensures that no command-generated results will be lost, even if the results are generated before you subscribe.
- toObservable(): When this method is called, the request is not executed immediately, but when a subscriber subscribes.
Fallback
In most cases, we want command to have a candidate method to handle failures, such as returning a default value or performing other fail-handling logic, except in the following cases:
- Write command: When the goal of a command is to perform a write operation rather than a read operation, it is usually necessary to hand over errors that fail to write to the caller.
- Batch processing system/offline computing: If the goal of command is to do some offline computing, generate reports, fill caches, etc., then the failure should also be handed over to the caller.
Whether or not the command implements the getFallback() method, the status of Hystrix and circuit-breaker will be updated when the command fails to execute.
HystrixCommand can degrade by implementing getFallback(). getFallback() is called when the run() method is abnormal, the execution timeout, the thread pool or semaphore is full, the service is refused, and the circuit breaker is short-circuited.
public class CommandHelloFailure extends HystrixCommand<String> { private final String name; public CommandHelloFailure(String name) { super(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")); this.name = name; } @Override protected String run() { throw new RuntimeException("this command always fails"); } @Override protected String getFallback() { return "Hello Failure " + name + "!"; } }
The run() method of this command always fails to execute, but the caller always receives the value returned by the getFallback() method instead of an exception:
@Test public void testSynchronous() { assertEquals("Hello Failure World!", new CommandHelloFailure("World").execute()); assertEquals("Hello Failure Bob!", new CommandHelloFailure("Bob").execute()); }
Hystrix Observable Command can return to another Observable when the original Observable fails by overloading the resumeWithFallback method. It should be noted that the original Observable may not produce errors until multiple results are issued. Therefore, it should not be assumed that the subscriber will only receive the results from the failed logic in the fallback implementation logic.
Hystrix uses RxJava internally onErrorResumeNext Operator to achieve seamless transfer between Observable s.
Error Propagation
Except for HystrixBadRequestException exceptions, all exceptions thrown in run methods are considered execution failures and trigger the logic of getFallback() methods and circuit breakers.
You can wrap the exception you want to throw in HystrixBadRequestException and get it through the getCause() method. HystrixBadRequestException uses scenarios where failure metrics should not be counted and getFallback() methods should not be triggered, such as reporting illegitimate parameters or non-system exceptions.
For Hystrix ObservableCommand, unrecoverable errors are notified by onError method and Observable returned by the user-implemented resumeWithFallback() method is used to complete the fallback mechanism.
Execution exception type
| Failure Type | Exception class | Exception.cause |
|---|---|---|
| FAILURE | HystrixRuntimeException | underlying exception(user-controlled) |
| TIMEOUT | HystrixRuntimeException | j.u.c.TimeoutException |
| SHORT_CIRCUITED | HystrixRuntimeException | j.l.RuntimeException |
| THREAD_POOL_REJECTED | HystrixRuntimeException | j.u.c.RejectedExecutionException |
| SEMAPHORE_REJECTED | HystrixRuntimeException | j.l.RuntimeException |
| BAD_REQUEST | HystrixBadRequestException | underlying exception(user-controller) |
Command Name
The default command name is derived from the class name:
getClass().getSimpleName()
You can specify the command name by using the constructor of HystrixCommand or HystrixObservableCommand:
public CommandHelloWorld(String name) { super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")) .andCommandKey(HystrixCommandKey.Factory.asKey("HelloWorld"))); this.name = name; }
Setter can be reused as follows:
private static final Setter cachedSetter = Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")) .andCommandKey(HystrixCommandKey.Factory.asKey("HelloWorld")); public CommandHelloWorld(String name) { super(cachedSetter); this.name = name; }
HystrixCommandKey is an interface, so it can be implemented as an enumeration or a regular class, but it has built-in a Factory class to help build internal instances in the following ways:
HystrixCommandKey.Factory.asKey("Hello World");
Command Group
Hystrix uses command group for grouping, and grouping information is mainly used for reporting, alerting, dashboard display, or identifying the owner of the team/library.
By default, Hystrix will use this name to define the thread pool of command unless a semaphore has been defined with this name.
HystrixCommandGroupKey is an interface, so it can be implemented as an enumeration or a regular class, but it has built-in a Factory class to help build internal instances in the following ways:
HystrixCommandGroupKey.Factory.asKey("Example Group")
Command Thread-pool
thread-pool key is mainly used to identify a HystrixThreadPool, a HystrixCommand associated with the HystrixThreadPool specified by the HystrixThreadPool Key passed into its constructor in similar scenarios such as monitoring, index publishing, caching, etc. If not specified, HystrixCommand Group Key is used to obtain/create HystrixThreadPool.
You can specify its value through the constructor of HystrixCommand or HystrixObservableCommand:
public CommandHelloWorld(String name) { super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")) .andCommandKey(HystrixCommandKey.Factory.asKey("HelloWorld")) .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey("HelloWorldPool"))); this.name = name; }
HystrixCommandThreadPoolKey is an interface, so it can be implemented as an enumeration or regular class, but it has built-in a Factory class to help build internal instances in the following ways:
HystrixThreadPoolKey.Factory.asKey("Hello World Pool")
The reason for using HystrixThreadPoolKey instead of using different HystrixCommandGroupKey is that there may be multiple commands that logically belong to the same group, but some of them need to be isolated from other commands, such as:
- Two command s for accessing video metadata
- Both command group name s are VideoMetadata
- command A is mutually exclusive with resource #1
- command B and Resource #2 Mutually Exclusive
If command A runs out of thread pool resources due to delays and other reasons, it should not affect command B's execution of # 2, because they access different back-end resources.
So logically, we want these two commands to be grouped together, but our system also separates the execution of these two commands, so we use Hystrix ThreadPoolKey to allocate them to different thread pools.
Request Cache
The caching of requests can be enabled by implementing the getCacheKey() method of HystrixCommand or HystrixObservableCommand:
public class CommandUsingRequestCache extends HystrixCommand<Boolean> { private final int value; protected CommandUsingRequestCache(int value) { super(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")); this.value = value; } @Override protected Boolean run() { return value == 0 || value % 2 == 0; } @Override protected String getCacheKey() { return String.valueOf(value); } }
Since this function depends on the context information of the request, we must initialize a HystrixRequestContext in the following way:
@Test public void testWithoutCacheHits() { HystrixRequestContext context = HystrixRequestContext.initializeContext(); try { assertTrue(new CommandUsingRequestCache(2).execute()); assertFalse(new CommandUsingRequestCache(1).execute()); assertTrue(new CommandUsingRequestCache(0).execute()); assertTrue(new CommandUsingRequestCache(58672).execute()); } finally { context.shutdown(); } }
Usually, context information (HystrixRequestContext) should be initialized and closed in ServletFilter or other classes with lifecycle management functions that hold user requests.
The following example shows how command retrieves data from caches and how to query whether a data is retrieved from caches: ___________
@Test public void testWithCacheHits() { HystrixRequestContext context = HystrixRequestContext.initializeContext(); try { CommandUsingRequestCache command2a = new CommandUsingRequestCache(2); CommandUsingRequestCache command2b = new CommandUsingRequestCache(2); assertTrue(command2a.execute()); // this is the first time we've executed this command with // the value of "2" so it should not be from cache assertFalse(command2a.isResponseFromCache()); assertTrue(command2b.execute()); // this is the second time we've executed this command with // the same value so it should return from cache assertTrue(command2b.isResponseFromCache()); } finally { context.shutdown(); } // start a new request context context = HystrixRequestContext.initializeContext(); try { CommandUsingRequestCache command3b = new CommandUsingRequestCache(2); assertTrue(command3b.execute()); // this is a new request context so this // should not come from cache assertFalse(command3b.isResponseFromCache()); } finally { context.shutdown(); } }
Request Collapsing
Request merging can be used to bind multiple requests together and execute by the same HystrixCommand instance.
collapser can automatically merge requests through batch size and the time consumed since batch was created.
Hystrix supports two ways of request merging: request level merging and global level merging. By default, request range merging can specify values when constructing collapser.
Request-scoped collapsers merge only requests in each HystrixRequestContext, whereas global-scoped collapsers merge requests across HystrixRequestContext. Therefore, if your downstream dependants cannot process multiple HystrixRequestContexts in another command, you should use request-level merges.
In Netflix, we only use request-level merging because all our current systems are built on the assumption that a command corresponds to a Hystrix RequestContext. Therefore, merging is effective when a command executes concurrently in a request with different parameters.
The following code shows how to implement Hystrix Collapser at the request level:
public class CommandCollapserGetValueForKey extends HystrixCollapser<List<String>, String, Integer> { private final Integer key; public CommandCollapserGetValueForKey(Integer key) { this.key = key; } @Override public Integer getRequestArgument() { return key; } @Override protected HystrixCommand<List<String>> createCommand(final Collection<CollapsedRequest<String, Integer>> requests) { return new BatchCommand(requests); } @Override protected void mapResponseToRequests(List<String> batchResponse, Collection<CollapsedRequest<String, Integer>> requests) { int count = 0; for (CollapsedRequest<String, Integer> request : requests) { request.setResponse(batchResponse.get(count++)); } } private static final class BatchCommand extends HystrixCommand<List<String>> { private final Collection<CollapsedRequest<String, Integer>> requests; private BatchCommand(Collection<CollapsedRequest<String, Integer>> requests) { super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")) .andCommandKey(HystrixCommandKey.Factory.asKey("GetValueForKey"))); this.requests = requests; } @Override protected List<String> run() { ArrayList<String> response = new ArrayList<String>(); for (CollapsedRequest<String, Integer> request : requests) { // artificial response for each argument received in the batch response.add("ValueForKey: " + request.getArgument()); } return response; } } }
The following code shows how to use collapser to automatically merge four CommandCollapserGetValueForKey into a HystrixCommand to execute:
@Test public void testCollapser() throws Exception { HystrixRequestContext context = HystrixRequestContext.initializeContext(); try { Future<String> f1 = new CommandCollapserGetValueForKey(1).queue(); Future<String> f2 = new CommandCollapserGetValueForKey(2).queue(); Future<String> f3 = new CommandCollapserGetValueForKey(3).queue(); Future<String> f4 = new CommandCollapserGetValueForKey(4).queue(); assertEquals("ValueForKey: 1", f1.get()); assertEquals("ValueForKey: 2", f2.get()); assertEquals("ValueForKey: 3", f3.get()); assertEquals("ValueForKey: 4", f4.get()); // assert that the batch command 'GetValueForKey' was in fact // executed and that it executed only once assertEquals(1, HystrixRequestLog.getCurrentRequest().getExecutedCommands().size()); HystrixCommand<?> command = HystrixRequestLog.getCurrentRequest().getExecutedCommands().toArray(new HystrixCommand<?>[1])[0]; // assert the command is the one we're expecting assertEquals("GetValueForKey", command.getCommandKey().name()); // confirm that it was a COLLAPSED command execution assertTrue(command.getExecutionEvents().contains(HystrixEventType.COLLAPSED)); // and that it was successful assertTrue(command.getExecutionEvents().contains(HystrixEventType.SUCCESS)); } finally { context.shutdown(); } }
Request Context Setup
When using request level features (such as request caching, request merging, request logs), you must manage the lifecycle of Hystrix RequestContext (or implement Hystrix Concurrent Stategy).
This means that you must execute the following code before requesting:
HystrixRequestContext context = HystrixRequestContext.initializeContext();
At the end of the request, the following code is executed:
context.shutdown();
In standard Java web applications, you can use Setvlet Filter to implement the following filters to manage:
public class HystrixRequestContextServletFilter implements Filter { public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { HystrixRequestContext context = HystrixRequestContext.initializeContext(); try { chain.doFilter(request, response); } finally { context.shutdown(); } } }
The following code can be added to web.xml to implement the use of this filter for all requests:
<filter> <display-name>HystrixRequestContextServletFilter</display-name> <filter-name>HystrixRequestContextServletFilter</filter-name> <filter-class>com.netflix.hystrix.contrib.requestservlet.HystrixRequestContextServletFilter</filter-class> </filter> <filter-mapping> <filter-name>HystrixRequestContextServletFilter</filter-name> <url-pattern>/*</url-pattern> </filter-mapping>
Common Patterns
Following are the general usage and usage patterns of HystrixCommand and HystrixObservableCommand.
Fail Fast
The most basic use is to execute a command that does only one thing and does not implement a fallback method. Such a command throws an exception when any error occurs:
public class CommandThatFailsFast extends HystrixCommand<String> { private final boolean throwException; public CommandThatFailsFast(boolean throwException) { super(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")); this.throwException = throwException; } @Override protected String run() { if (throwException) { throw new RuntimeException("failure from CommandThatFailsFast"); } else { return "success"; } }
The following code demonstrates the above behavior:
@Test public void testSuccess() { assertEquals("success", new CommandThatFailsFast(false).execute()); } @Test public void testFailure() { try { new CommandThatFailsFast(true).execute(); fail("we should have thrown an exception"); } catch (HystrixRuntimeException e) { assertEquals("failure from CommandThatFailsFast", e.getCause().getMessage()); e.printStackTrace(); } }
HystrixObservableCommand needs to overload the resumeWithFallback() method to achieve the same behavior:
@Override protected Observable<String> resumeWithFallback() { if (throwException) { return Observable.error(new Throwable("failure from CommandThatFailsFast")); } else { return Observable.just("success"); } }
Fail Silent
Silent failure is equivalent to returning an empty response or removal function. It can be returning null, empty Map, empty List, or other similar response.
This can be achieved by implementing the HystrixCommand.getFallback() method:
public class CommandThatFailsSilently extends HystrixCommand<String> { private final boolean throwException; public CommandThatFailsSilently(boolean throwException) { super(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")); this.throwException = throwException; } @Override protected String run() { if (throwException) { throw new RuntimeException("failure from CommandThatFailsFast"); } else { return "success"; } } @Override protected String getFallback() { return null; } }
@Test public void testSuccess() { assertEquals("success", new CommandThatFailsSilently(false).execute()); } @Test public void testFailure() { try { assertEquals(null, new CommandThatFailsSilently(true).execute()); } catch (HystrixRuntimeException e) { fail("we should not get an exception as we fail silently with a fallback"); } }
Or return an empty List as follows:
@Override protected List<String> getFallback() { return Collections.emptyList(); }
HystrixObservableCommand can achieve the same behavior by overloading resumeWithFallback():
@Override protected Observable<String> resumeWithFallback() { return Observable.empty(); }
Fallback: Static
Fallback can return the default values set in the code, which can effectively avoid the impact of silent failure by default behavior.
For example, if a user-authenticated command that should return true/false fails to execute, its default behavior can be as follows:
@Override protected Boolean getFallback() { return true; }
The same behavior can be achieved by overloading the resumeWithFallback() method for HystrixObservableCommand:
@Override protected Observable<Boolean> resumeWithFallback() { return Observable.just( true ); }
Fallback: Stubbed
When command returns a composite object with multiple fields, and part of the field value of the object can be obtained by other request states, and the other part of the state can be obtained by setting default values, you usually need to use stubbed mode.
You may get the appropriate values from stubbed values as follows:
- cookies
- Request parameters and request headers
- Response to the previous service request of the current failed request
The stubbed values within the fallback block can be retrieved statically, but usually we recommend injecting these values when building command instances, just like countryCodeFromGeoLookup in the code of the following example:
public class CommandWithStubbedFallback extends HystrixCommand<UserAccount> { private final int customerId; private final String countryCodeFromGeoLookup; /** * @param customerId * The customerID to retrieve UserAccount for * @param countryCodeFromGeoLookup * The default country code from the HTTP request geo code lookup used for fallback. */ protected CommandWithStubbedFallback(int customerId, String countryCodeFromGeoLookup) { super(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")); this.customerId = customerId; this.countryCodeFromGeoLookup = countryCodeFromGeoLookup; } @Override protected UserAccount run() { // fetch UserAccount from remote service // return UserAccountClient.getAccount(customerId); throw new RuntimeException("forcing failure for example"); } @Override protected UserAccount getFallback() { /** * Return stubbed fallback with some static defaults, placeholders, * and an injected value 'countryCodeFromGeoLookup' that we'll use * instead of what we would have retrieved from the remote service. */ return new UserAccount(customerId, "Unknown Name", countryCodeFromGeoLookup, true, true, false); } public static class UserAccount { private final int customerId; private final String name; private final String countryCode; private final boolean isFeatureXPermitted; private final boolean isFeatureYPermitted; private final boolean isFeatureZPermitted; UserAccount(int customerId, String name, String countryCode, boolean isFeatureXPermitted, boolean isFeatureYPermitted, boolean isFeatureZPermitted) { this.customerId = customerId; this.name = name; this.countryCode = countryCode; this.isFeatureXPermitted = isFeatureXPermitted; this.isFeatureYPermitted = isFeatureYPermitted; this.isFeatureZPermitted = isFeatureZPermitted; } } }
The following code demonstrates the above behavior:
@Test public void test() { CommandWithStubbedFallback command = new CommandWithStubbedFallback(1234, "ca"); UserAccount account = command.execute(); assertTrue(command.isFailedExecution()); assertTrue(command.isResponseFromFallback()); assertEquals(1234, account.customerId); assertEquals("ca", account.countryCode); assertEquals(true, account.isFeatureXPermitted); assertEquals(true, account.isFeatureYPermitted); assertEquals(false, account.isFeatureZPermitted); }
The same behavior can be achieved by overloading the resumeWithFallback() method for HystrixObservableCommand:
@Override protected Observable<Boolean> resumeWithFallback() { return Observable.just( new UserAccount(customerId, "Unknown Name", countryCodeFromGeoLookup, true, true, false) ); }
If you want to emit multiple values from Observable, then when failure occurs, the original Observable may have emitted part of the value. At this time, you might prefer to be able to emit only another part of the value that has not been emitted from fallback logic. The following example shows how to achieve this goal: it achieves fallback logic by tracking the last value emitted by the original Observable. Where should Observable continue to issue stubbed values in the album?
@Override protected Observable<Integer> construct() { return Observable.just(1, 2, 3) .concatWith(Observable.<Integer> error(new RuntimeException("forced error"))) .doOnNext(new Action1<Integer>() { @Override public void call(Integer t1) { lastSeen = t1; } }) .subscribeOn(Schedulers.computation()); } @Override protected Observable<Integer> resumeWithFallback() { if (lastSeen < 4) { return Observable.range(lastSeen + 1, 4 - lastSeen); } else { return Observable.empty(); } }
Fallback: Cache via Network
Sometimes the command execution fails because of the back-end service exception. At this time, we can also get the relevant data from the cache (e.g. memcached).
Since access to the network in fallback's logical code may fail again, it is necessary to build a new HystrixCommand or HystrixObservableCommand to execute:
It is important that commands that execute fallback logic need to be executed in a different thread pool. Otherwise, commands that execute fallback logic cannot execute in the same thread pool if the delay of the original command increases and the thread pool is full.
The following code shows how CommandWithFallbackViaNetwork executes FallbackViaNetwork in the getFallback() method.
Note that FallbackViaNetwork also has a fallback mechanism, where fail silent is achieved by returning null.
FallbackViaNetwork inherits the thread pool configuration RemoteService X from HystrixCommand Group Key by default, so you need to inject HystrixThreadPoolKey. Factory. asKey ("RemoteService X Fallback") into its constructor to execute it in different thread pools.
In this way, Command With Fallback ViaNetwork will execute in a thread pool named RemoteService X, while Fallback ViaNetwork will execute in a thread pool named RemoteService X Fallback.
public class CommandWithFallbackViaNetwork extends HystrixCommand<String> { private final int id; protected CommandWithFallbackViaNetwork(int id) { super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("RemoteServiceX")) .andCommandKey(HystrixCommandKey.Factory.asKey("GetValueCommand"))); this.id = id; } @Override protected String run() { // RemoteServiceXClient.getValue(id); throw new RuntimeException("force failure for example"); } @Override protected String getFallback() { return new FallbackViaNetwork(id).execute(); } private static class FallbackViaNetwork extends HystrixCommand<String> { private final int id; public FallbackViaNetwork(int id) { super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("RemoteServiceX")) .andCommandKey(HystrixCommandKey.Factory.asKey("GetValueFallbackCommand")) // use a different threadpool for the fallback command // so saturating the RemoteServiceX pool won't prevent // fallbacks from executing .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey("RemoteServiceXFallback"))); this.id = id; } @Override protected String run() { MemCacheClient.getValue(id); } @Override protected String getFallback() { // the fallback also failed // so this fallback-of-a-fallback will // fail silently and return null return null; } } }
Primary + Secondary with Fallback
Some systems may have a dual-system model-master-slave mode or master-backup mode.
Sometimes a slave or standby system is considered to be a failure state, which is used only when fallback logic is executed; this scenario is the same as the scenario described in the Cache via Network section.
However, if switching to slave system is normal, for example, when new code is released (which is a way for stateful system to publish code), the main system will be unavailable whenever switching to slave system, and the circuit breaker will open and alert.
This is not what we expect. This wolf-like alarm may lead to real problems that we ignore as normal false alarms.
Therefore, by placing a facade Hystrix Command in front of it (see below), we can treat the master/slave system switching as a normal and healthy state.
Master-slave HystrixCommand needs to access the network and implement specific business logic, so its implementation should be thread-isolated. They may have significant performance gaps (usually the slave system is a static cache), so another advantage of isolating the two command s is that they can be tailored.
You don't need to publish both commands publicly. You just need to hide them in another semaphore-isolated HistrixCommand (called Facade HystrixCommand), in which to implement the call selection between master and slave systems. Only when both master and slave systems fail, will the fallback logic of the facade command be implemented.
Facade HystrixCommand can use semaphore isolation, because its business logic only calls the other two threads isolated HistrixCommand, it does not involve any network access, retry and other error-prone things, so it is not necessary to put this part of the code to other threads to execute.
public class CommandFacadeWithPrimarySecondary extends HystrixCommand<String> { private final static DynamicBooleanProperty usePrimary = DynamicPropertyFactory.getInstance().getBooleanProperty("primarySecondary.usePrimary", true); private final int id; public CommandFacadeWithPrimarySecondary(int id) { super(Setter .withGroupKey(HystrixCommandGroupKey.Factory.asKey("SystemX")) .andCommandKey(HystrixCommandKey.Factory.asKey("PrimarySecondaryCommand")) .andCommandPropertiesDefaults( // we want to default to semaphore-isolation since this wraps // 2 others commands that are already thread isolated HystrixCommandProperties.Setter() .withExecutionIsolationStrategy(ExecutionIsolationStrategy.SEMAPHORE))); this.id = id; } @Override protected String run() { if (usePrimary.get()) { return new PrimaryCommand(id).execute(); } else { return new SecondaryCommand(id).execute(); } } @Override protected String getFallback() { return "static-fallback-" + id; } @Override protected String getCacheKey() { return String.valueOf(id); } private static class PrimaryCommand extends HystrixCommand<String> { private final int id; private PrimaryCommand(int id) { super(Setter .withGroupKey(HystrixCommandGroupKey.Factory.asKey("SystemX")) .andCommandKey(HystrixCommandKey.Factory.asKey("PrimaryCommand")) .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey("PrimaryCommand")) .andCommandPropertiesDefaults( // we default to a 600ms timeout for primary HystrixCommandProperties.Setter().withExecutionTimeoutInMilliseconds(600))); this.id = id; } @Override protected String run() { // perform expensive 'primary' service call return "responseFromPrimary-" + id; } } private static class SecondaryCommand extends HystrixCommand<String> { private final int id; private SecondaryCommand(int id) { super(Setter .withGroupKey(HystrixCommandGroupKey.Factory.asKey("SystemX")) .andCommandKey(HystrixCommandKey.Factory.asKey("SecondaryCommand")) .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey("SecondaryCommand")) .andCommandPropertiesDefaults( // we default to a 100ms timeout for secondary HystrixCommandProperties.Setter().withExecutionTimeoutInMilliseconds(100))); this.id = id; } @Override protected String run() { // perform fast 'secondary' service call return "responseFromSecondary-" + id; } } public static class UnitTest { @Test public void testPrimary() { HystrixRequestContext context = HystrixRequestContext.initializeContext(); try { ConfigurationManager.getConfigInstance().setProperty("primarySecondary.usePrimary", true); assertEquals("responseFromPrimary-20", new CommandFacadeWithPrimarySecondary(20).execute()); } finally { context.shutdown(); ConfigurationManager.getConfigInstance().clear(); } } @Test public void testSecondary() { HystrixRequestContext context = HystrixRequestContext.initializeContext(); try { ConfigurationManager.getConfigInstance().setProperty("primarySecondary.usePrimary", false); assertEquals("responseFromSecondary-20", new CommandFacadeWithPrimarySecondary(20).execute()); } finally { context.shutdown(); ConfigurationManager.getConfigInstance().clear(); } } } }
Client Doesn't Perform Network Access
When you use HystrixCommand to implement business logic that does not involve network access, latency-sensitive, and unacceptable multithreading overhead, you need to set up executionIsolationStrategy ) The value of the attribute is ExecutionIsolationStrategy SEMAPHORE, Hystrix will use semaphore isolation instead of thread isolation.
The following code shows how to set this property for command (you can also dynamically change the value of this property at run time):
public class CommandUsingSemaphoreIsolation extends HystrixCommand<String> { private final int id; public CommandUsingSemaphoreIsolation(int id) { super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")) // since we're doing an in-memory cache lookup we choose SEMAPHORE isolation .andCommandPropertiesDefaults(HystrixCommandProperties.Setter() .withExecutionIsolationStrategy(ExecutionIsolationStrategy.SEMAPHORE))); this.id = id; } @Override protected String run() { // a real implementation would retrieve data from in memory data structure return "ValueFromHashMap_" + id; } }
Get-Set-Get with Request Cache Invalidation
Get-Set-Get means that when the result of the Get request is cached, another command sends a Set request to the same resource. At this time, the result cached by the Get request should be invalid, so as to avoid the subsequent Get request getting the outdated cached result, which can be called at this time. HystrixRequestCache.clear() ) Method to invalidate the cache.
public class CommandUsingRequestCacheInvalidation { /* represents a remote data store */ private static volatile String prefixStoredOnRemoteDataStore = "ValueBeforeSet_"; public static class GetterCommand extends HystrixCommand<String> { private static final HystrixCommandKey GETTER_KEY = HystrixCommandKey.Factory.asKey("GetterCommand"); private final int id; public GetterCommand(int id) { super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("GetSetGet")) .andCommandKey(GETTER_KEY)); this.id = id; } @Override protected String run() { return prefixStoredOnRemoteDataStore + id; } @Override protected String getCacheKey() { return String.valueOf(id); } /** * Allow the cache to be flushed for this object. * * @param id * argument that would normally be passed to the command */ public static void flushCache(int id) { HystrixRequestCache.getInstance(GETTER_KEY, HystrixConcurrencyStrategyDefault.getInstance()).clear(String.valueOf(id)); } } public static class SetterCommand extends HystrixCommand<Void> { private final int id; private final String prefix; public SetterCommand(int id, String prefix) { super(HystrixCommandGroupKey.Factory.asKey("GetSetGet")); this.id = id; this.prefix = prefix; } @Override protected Void run() { // persist the value against the datastore prefixStoredOnRemoteDataStore = prefix; // flush the cache GetterCommand.flushCache(id); // no return value return null; } } }
@Test public void getGetSetGet() { HystrixRequestContext context = HystrixRequestContext.initializeContext(); try { assertEquals("ValueBeforeSet_1", new GetterCommand(1).execute()); GetterCommand commandAgainstCache = new GetterCommand(1); assertEquals("ValueBeforeSet_1", commandAgainstCache.execute()); // confirm it executed against cache the second time assertTrue(commandAgainstCache.isResponseFromCache()); // set the new value new SetterCommand(1, "ValueAfterSet_").execute(); // fetch it again GetterCommand commandAfterSet = new GetterCommand(1); // the getter should return with the new prefix, not the value from cache assertFalse(commandAfterSet.isResponseFromCache()); assertEquals("ValueAfterSet_1", commandAfterSet.execute()); } finally { context.shutdown(); } } }
Migrating a Library to Hystrix
If you want to migrate an existing client library to Hystrix, you should replace all service methods with HystrixCommand.
The service methods call HystrixCommand instead of containing any additional business logic.
Therefore, before migration, a service library might be like this:
After migration, users of the service library should have direct access to HystrixCommand or indirect access to HystrixCommand through service facade agents.
