feat!: allow registering multiple encoders in Feign#3476
Conversation
Co-authored-by: trumpetinc <6618744+trumpetinc@users.noreply.github.com>
|
I think this could be accomplished by a single encoder that accepts multiple encoders. The blast radius of the chance would also be much smaller |
|
@velo I agree. I'm just wondering what would be better from user perspective: var client = Feign.builder()
.encoder(Encoder.of(new Encoder1(), new Encoder2()))
.target(Client.class, "https://example.com");
// or
var client = Feign.builder()
.encoders(new Encoder1(), new Encoder2())
.target(Client.class, "https://example.com"); |
Co-authored-by: trumpetinc <6618744+trumpetinc@users.noreply.github.com>
Co-authored-by: trumpetinc <6618744+trumpetinc@users.noreply.github.com>
Co-authored-by: trumpetinc <6618744+trumpetinc@users.noreply.github.com>
|
The build fails because |
|
@yvasyliev can you please take a look at the changes required to have a separate canEncode() method vs changing the return type to boolean? I think you will find that the Encoder implementations are going to wind up with a bunch of duplicate code... Take DefaultEncoder as an example. If we change the return type of encode(), then the implementation is this: If we have to have a separate method, then we wind up with this: That's a ton of duplication. I understand the reason for the separate method (so you can use an interface with default method as a mix-in), but is it worth it? I just feel like adding the check to the top of the handful of effected encoders is simpler (and you are already having to modify those encoders to add the marker interface)... If we want to have common "this is a JSON content-type" functionality, then let's put that in a utility class and add a check for it at the top of the various json encoders' boolean encode() method... (or use the PredicateEncoder design - but I can see why we might prefer to have the json and xml content types automatically handled). |
|
I still think that the PredicateEncoder design is appropriate for scenarios where we want to tune encoders based on content-type headers, etc... |
I think that we could go with the top option for now if it makes it easier to get this change accepted. But it would be very straightforward to make the current configbuilder.encoder() method be vararg as a syntactic sugar improvement. All it would do would be to check the length of the vararg, and if >1 it would implicitly do the Encoder.of(...) call. Internally, Feign will continue to only use a single encoder. This would be source (but not binary) compatible - anyone currently passing a single argument would get the exact same experience. |
| * @return this builder | ||
| * @since 14 | ||
| */ | ||
| public B encoders(List<Encoder> encoders) { |
There was a problem hiding this comment.
I think if we replace encoder(Encoder) with encoder(Encoder...) and encoder(List<Encoder>) that we can eliminate the encoders() method.
This will drastically reduce the impact on all the test classes, etc...
I believe a valid implementation would look like this: public class DefaultEncoder implements Encoder {
@Override
public boolean encode(Object object, Type bodyType, RequestTemplate template) {
if (bodyType == String.class) {
template.body(Request.Body.of(object.toString()));
return true;
} else if (bodyType == byte[].class) {
template.body(Request.Body.of((byte[]) object));
return true;
} else if (object instanceof File) {
template.body(new Request.PathBody(((File) object).toPath()));
return true;
} else if (object instanceof Path) {
template.body(new Request.PathBody((Path) object));
return true;
} else if (object instanceof InputStream) {
template.body(new Request.InputStreamBody((InputStream) object));
return true;
} else if (object instanceof Request.Body) {
template.body((Request.Body) object);
return true;
} else {
return false;
}
}
}
Is it relevant for
Won't we end up with duplicated I decided to give the abstract
My bet is that the |
|
This is splitting hairs, but here is how I would implement that logic to reduce lines of code: |
I think that we are talking doubling the lines of code (and implementing the same logic in two methods instead of one). So no, I think this is relevant across the board. Spreading logic across two methods dramatically increases the chances of mistakes (and these are the types of nasty mistakes that will manifest as weird corner cases). Plus this will make migrating existing code that much more complicated. Fundamentally, though, there is no significant upside to doing this. The default method/mix-in interface approach is clever - but it isn't doing very much. Instead, we could break the content type check out into a static isContentTypeJson() utility method, then call that utility method at the top of the 4 JSON encoders. This is just as many lines of code as adding the JsonEncoder interface to each of the 4 JSON encoders. Like this: And Jackson encoding would look like this: So for the json and xml cases, we are talking about whether we want to add 'JsonEncoder' to the implements line, or whether we just add one line to the top of the encode() method. I just don't think the clutter is worth it in all of the other encoder types. It just seems like it is better for our json and xml encoders to have one extra line of code than force all the other encoders to double their complexity. Unit testing the utility class is more straightforward, etc...
Yep - just like we have duplicated 'implements JsonEncoder' in multiple encoders. I feel that line of code is trivial to add where it is needed. I just don't think the default method is helping us. I do acknowledge that that if there a ton of existing encoders that already implement JsonEncoder, then having the extra method would theoretically make these encoders so they work as-is without making changes to them. So it's a tradeoff: Do we:
I am encouraging that we pick #2 because it improves the experience for ALL encoders. Quasi-related: Are we clear on why JsonEncoder even exists? I can't figure out any real-world reason for that interface to even be there. |
This is a good point that I hadn't considered. You've worked the test code base much more than me - how bad is it? Would it make the unit tests an absolute nightmare to overhaul? If so, then that is an additional item that needs to be accounted for in the tradeoff analysis. PS - the reason you can't return Request.Body is that the encode() method is expected to be able to make changes to the RequestTemplate (adding headers, etc...). Encoding isn't just about creating a body. So returning true/false is what makes sense to me. It indicates whether the method actually modified the RequestTemplate it was passed. |
|
Oh - and at the end of the day, the decision on this doesn't impact my excitement about the direction. I'm just providing architectural feedback based on a long, long time of designing and working with these sorts of systems. Any time related business logic is implemented in multiple places, it increases the risk of really nasty corner case bugs. I have lost many, many weeks in my career diagnosing these sorts of bugs, so I am very resistant :-) |
|
You raise a fair point about duplication — I want to explore whether it's actually the architecture or just I'm not convinced that
public class StringEncoder implements Encoder {
@Override
public boolean canEncode(Object object, Type bodyType, RequestTemplate template) {
return bodyType == String.class;
}
@Override
public void encode(Object object, Type bodyType, RequestTemplate template) {
template.body(Request.Body.of(object.toString()));
}
}
public class ByteArrayEncoder implements Encoder {
@Override
public boolean canEncode(Object object, Type bodyType, RequestTemplate template) {
return bodyType == byte[].class;
}
@Override
public void encode(Object object, Type bodyType, RequestTemplate template) {
template.body(Request.Body.of((byte[]) object));
}
}
public class FileEncoder implements Encoder {
@Override
public boolean canEncode(Object object, Type bodyType, RequestTemplate template) {
return object instanceof File;
}
@Override
public void encode(Object object, Type bodyType, RequestTemplate template) {
template.body(new Request.PathBody(((File) object).toPath()));
}
}
public class PathEncoder implements Encoder {
@Override
public boolean canEncode(Object object, Type bodyType, RequestTemplate template) {
return object instanceof Path;
}
@Override
public void encode(Object object, Type bodyType, RequestTemplate template) {
template.body(new Request.PathBody((Path) object));
}
}
public class InputStreamEncoder implements Encoder {
@Override
public boolean canEncode(Object object, Type bodyType, RequestTemplate template) {
return object instanceof InputStream;
}
@Override
public void encode(Object object, Type bodyType, RequestTemplate template) {
template.body(new Request.InputStreamBody((InputStream) object));
}
}
public class RequestBodyEncoder implements Encoder {
@Override
public boolean canEncode(Object object, Type bodyType, RequestTemplate template) {
return object instanceof Request.Body;
}
@Override
public void encode(Object object, Type bodyType, RequestTemplate template) {
template.body((Request.Body) object);
}
}Do any of the encoders above have code duplication? Are any of them error-prone? I'd argue that we shouldn't |
|
I really appreciate this discussion. I often work on instinct, and the back and forth is encouraging me to really think hard about why I am uncomfortable with the canEncode() approach. I will try to lay my concrete reasons here in several sections. Semantic change to encode() - not just a method signature change The fundamental issue is that we are not just changing the API signature - we are literally changing the semantics of the encode() method.
Unfortunately, it is not possible to bridge a semantic change like this with a default method. The encoder's encode() logic must be changed so it does not throw an exception when it can't handle the request. So it is really important that we break existing code and force a rewrite of every encoder. This may sound like a small thing, but it is actually quite big. Right now, you are assuming that checking the content-type header is sufficient to determine whether a given existing JsonEncoder or XmlEncoder can handle the encode. That might be true - but it also might not be true - we really have no way of knowing without actually reviewing the code of every JsonEncoder. Maybe there is some Json handler that explicitly doesn't handle certain types of situations? Also, Encoders can be written by 3rd parties, so we can't be the ones to check to make sure that the semantic change is fully realized by the default method. The default method approach could result in us calling encode when we shouldn't (and getting an EncodeException instead of being able to continue walking the encoder chain). We have now introduced a breaking change that is only detected at runtime. This is my biggest concern. Are Existing Encoders Really Impacted That Much By the Extra Method? I agree that the "normal" encoder examples you provided aren't that big of a deal (it does increase the LoCs a bit, though). Here are some examples where having the separate method is awkward: FormEncoder (I know the point of all of our work is to deprecate that, but it's a good example of why we need to be super careful) PredicateEncoder is also going to be more awkward - instead of just calling the delegate, now we have to check both our predicate and whether the delegate can encode. The new DelegatingEncoder is also awkward (though not as bad as the ones above) Interface Default Method Antipattern Using default methods in this way feels like a code smell. I admit that it is not 100% certain for this specific case, but documented anti-patterns exist for a reason, and we should have very, very good reasons for using an anti-pattern. I'm not closed minded on using antipatterns where it makes sense - but I'm also not yet convinced that I've seen big enough reasons to use an antipattern in this case. On the other side of the argument, we could say that the default method is there to bridge an API as it has evolved so we get reasonable default behavior for all of the existing implementations. And I would be ok with that if we were adding the default method to the Encoder interface (not possible, of course, b/c this is a semantic change, not something that a default method can handle). But even if there weren't semantic differences, it is awkward that we are evolving the Encoder interface, but adding the default method to the JsonEncoder sub-interface. I get that JsonEncoder extends Encoder - but it just feels wrong. Especially when there is an easy way to do this that isn't opaque. My primary objection on this is that using interface default methods in this way feels very opaque. It is not at all obvious looking at the code that there is special handling going on for the content-type. If we were using abstract base classes, then I wouldn't object - but interfaces (and especially what has traditionally been a marker interface - no behavior) really feel non-obvious. Conclusion I think that recognizing there there is a significant semantic change is a very good reason to not use a default method to try to bridge this change. We must force compile time failures to ensure that each encoder migrates to the new semantics. If that is the case, then I think the discussion becomes purely about whether having two methods is better than one method, and I definitely think that one method is better - but I could be convinced otherwise if the amount of code required to add a return type would be excessive (I think you said that there are a lot of lambdas in the unit tests??). If that does turn out to be the case, I would argue that introducing another exception type would still be better than a second method (but I'm not super crazy about that either - exceptions for standard control flow are ugly). Please let me know if it would be helpful for me to dive deeper into the semantic changes being made. This is subtle - but super important. |
Co-authored-by: trumpetinc <6618744+trumpetinc@users.noreply.github.com>
|
@trumpetinc |
|
One thing that I thought of while doing the code review: Users may be relying on the various json and xml encoders "just working" without having to set the content-type header... Or maybe they are intentionally setting a different content-type header for their specific endpoint? It is unfortunate that the encoders are created using constructors (if they were created using static factory methods we could use composition and wrap with a PredicateEncoder). I'm not sure what the real world risk on this is - but it could impact regular users (not just custom Encoder creators), so I think worth a quick discussion. It's been a long time since I've seen a REST endpoint that didn't work without the content-type set - so it seems unlikely that this would impact many existing users (if any). If we wanted to eliminate this, we could NOT do the content type check in the various json and xml encoders, but rather wrap them in a PredicateEncoder when needed. That would ensure complete backwards compatibility at the cost of more difficult DelegateExtractor setup. |
trumpetinc
left a comment
There was a problem hiding this comment.
This is an excellent PR in my opinion. My only feedback were small things on javadocs - very well done.
| * @param bodyType the type the object should be encoded as. {@link #MAP_STRING_WILDCARD} | ||
| * indicates form encoding. | ||
| * @param template the request template to populate. | ||
| * @return {@code true} if the encoder handled the object, {@code false} otherwise. |
There was a problem hiding this comment.
Suggested wording:
@return {@code true} if the encoder is able to handle the encode request, {@code false} if the encoder cannot encode the request
@throws EncodeException if this encoder should be able to encode the request but encoding fails
| try { | ||
| encoder.encode(formVariables, Encoder.MAP_STRING_WILDCARD, mutable); | ||
| if (!encoder.encode(formVariables, Encoder.MAP_STRING_WILDCARD, mutable)) { | ||
| throw new EncodeException("This encoder does not support form encoding: " + encoder); |
There was a problem hiding this comment.
Should we refer to the original form encoding as "legacy form encoding" maybe?
I'm a little concerned that it could be confusing to users if we don't distinguish between the two form encoding options.
| return super.resolve(argv, mutable, variables); | ||
| } | ||
|
|
||
| private void encode(Object object, Type bodyType, RequestTemplate mutable) |
There was a problem hiding this comment.
Consider renaming to encodeOrThrow
| } | ||
|
|
||
| // TODO: enable when Spring Cloud OpenFiegn migrates to Feign 14 | ||
| @Disabled("Disabled util Spring Cloud OpenFiegn migrates to Feign 14") |
There was a problem hiding this comment.
spelling - change 'util' to 'until' (add an 'n'); change 'OpenFiegn' to 'OpenFeign' (reverse i and e)
| (e.g., `application/json`, `application/ld+json`). | ||
| - XML encoders (JAXB, SOAP) check `Util.isXmlContentType(template)` and return | ||
| `false` when it does not match (e.g., `text/xml`, `application/xml`). | ||
|
|
There was a problem hiding this comment.
I think we probably need to have something along these lines:
Built in JSON and XML encoders require Content-Type header to be set:
If you are using json or xml encoding without setting the content-type header (either via annotations or a Feign request listener), you must add an appropriate content-type header (application/json, text/xml, application/xml).
Note that Spring Feign sets the content-type header to application/json by default, so this note does not apply to Spring Feign users).
There was a problem hiding this comment.
PS - this is a big enough impact that we may want to have it be a separately numbered migration line (i.e. #15).
Most of the other changes here do not directly impact normal users - but this change does.
|
Do we have to do anything with SpringEncoder or PageableSpringEncoder? Those are the only 2 Encoders that I'm seeing in the code base that aren't yet covered by this PR... |
Co-authored-by: trumpetinc 6618744+trumpetinc@users.noreply.github.com
Note
Target branch:
14.xImportant
Please squash and merge this PR if it's approved.
Important
Please add
Co-authored-by: trumpetinc <6618744+trumpetinc@users.noreply.github.com>footer to the commit message if this PR is approved.Summary
This PR introduces multi-encoder support in Feign, allowing users to register multiple encoders that are automatically selected at request time. The
Encoder.encode()method now returnsboolean(trueif the encoder handled the object,falseotherwise), and a newEncoder.of()factory combines encoders into aDelegatingEncoder.Motivation
Previously, Feign only supported a single
Encoderregistered viaFeign.builder().encoder(...). For APIs that mix different content types (JSON, XML, form data, etc.), users had to manually wrap encoders (e.g.,new FormEncoder(new JacksonEncoder())). This was brittle and didn't scale well.Changes
New API
Encoder.encode()now returnsboolean— returntruewhen encoding succeeds,falseif the encoder does not handle the request. This replaces the need for a separatecanEncode()method.Encoder.of(Encoder...)/Encoder.of(List<Encoder>)— factory method that creates aDelegatingEncoder, which tries each delegate'sencode()in order and uses the first one that returnstrue. ThrowsEncodeExceptionif no encoder succeeds.DelegatingEncoder— a composite encoder that delegates to a list of encoders, using the first one whoseencode()returnstrue.Util.isJsonContentType(RequestTemplate)/Util.isXmlContentType(RequestTemplate)— utility methods that check theContent-Typeheader against JSON (application/json,application/ld+json, etc.) and XML (text/xml,application/xml, etc.) patterns.Encoder Updates
All built-in encoders now return
booleanfromencode()and gate onContent-Typewhere applicable:DefaultEncoderString,byte[],File,Path,InputStream,Request.Body, andnullFormEncodermultipart/form-dataandapplication/x-www-form-urlencodedGraphqlEncoderGraphqlContract; delegates to wrapped encoder otherwiseMeteredEncoder(all variants)Util.isJsonContentType(template)Util.isXmlContentType(template)Example Usage
Before:
After:
Breaking Changes
Breaking changes are documented in
MIGRATION-v14.md(sections 14–18):Encoder.encode()now returnsbooleaninstead ofvoid. All customEncoderimplementations must update the return type and returntrueon success orfalsewhen the encoder does not handle the request.Feign.builder().encoder(new JacksonEncoder())still works.Files Changed
api/src/main/java/feign/codec/DelegatingEncoder.javaEncoder.java,Util.java,Feign.java,AsyncFeign.java,HystrixFeign.java,VertxFeign.java,DefaultEncoder.java,FormEncoder.java,GraphqlEncoder.java,MeteredEncoder.java(×3),JAXBEncoder.java(×2),SOAPEncoder.java(×2),JacksonEncoder.java,Jackson3Encoder.java,GsonEncoder.java,MoshiEncoder.java,Fastjson2Encoder.java,JacksonJrEncoder.java,JacksonJaxbJsonEncoder.javaDelegatingEncoderTest.java, all JSON/XML encoder test files, and various integration testsMIGRATION-v14.md,CHANGELOG.md