Spring 5.0.3 RequestRejectedException: the request was rejected because the URL was not normalized

Not sure if this is a bug in Spring 5.0.3 or a new feature to fix the situation on my part.

After the update, I get this error. Interestingly, this error is only on my local machine. The same code works fine in a test environment with HTTPS.

To be continued ...

The reason I get this error is because my URL to load the resulting JSP page is /location/thisPage.jsp . Evaluating the request.getRequestURI() code gives me the result of /WEB-INF/somelocation//location/thisPage.jsp . If I fix the JSP page URL in this location/thisPage.jsp , everything will be fine.

So my question is should I remove / from the JSP paths in the code, because that is what is required in the future. Or Spring made a mistake, because the only difference between my machine and the test environment is the HTTP against HTTPS .

  org.springframework.security.web.firewall.RequestRejectedException: The request was rejected because the URL was not normalized. at org.springframework.security.web.firewall.StrictHttpFirewall.getFirewalledRequest(StrictHttpFirewall.java:123) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:194) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270) 
+72
spring-mvc spring-security
source share
8 answers

The problem is planned to be solved in 5.1.0; Currently 5.0.0 does not have this problem.

0
source share

Spring Security Documentation mentions the reason for blocking // in the request.

For example, it may contain path traversal sequences (for example, / .. /) or several forward slashes (//), which may also cause pattern matching to fail. Some containers normalize them before performing a servlet mapping, but others do not. To protect against such problems, FilterChainProxy uses the HttpFirewall strategy to validate and transfer the request. Unnormalized requests are automatically rejected by default, and path parameters and duplicate slashes are deleted for matching.

So there are two possible solutions -

  1. remove double slash (preferred approach)
  2. Allow // in Spring Security by configuring StrictHttpFirewall using the code below.

Step 1 Create your own firewall that allows you to use slashes in URLs.

 @Bean public HttpFirewall allowUrlEncodedSlashHttpFirewall() { StrictHttpFirewall firewall = new StrictHttpFirewall(); firewall.setAllowUrlEncodedSlash(true); return firewall; } 

Step 2 And then configure this component in web security

 @Override public void configure(WebSecurity web) throws Exception { //@formatter:off super.configure(web); web.httpFirewall(allowUrlEncodedSlashHttpFirewall()); .... } 

Step 2 is optional, Spring Boot just needs to declare a bean like HttpFirewall

+54
source share

setAllowUrlEncodedSlash(true) did not work for me. The still internal isNormalized method returns false if there is a double slash.

I replaced StrictHttpFirewall with DefaultHttpFirewall , adding only the following code:

 @Bean public HttpFirewall defaultHttpFirewall() { return new DefaultHttpFirewall(); } 

Works well for me.
Is there any risk when using DefaultHttpFirewall ?

+19
source share

I ran into the same problem with:

Spring Boot Version = 1.5.10
Spring Security Version = 4.2.4


The problem arose at the endpoints where ModelAndView ModelAndView was defined with a previous slash. Example:

 ModelAndView mav = new ModelAndView("/your-view-here"); 

If I removed the slash, everything worked fine. Example:

 ModelAndView mav = new ModelAndView("your-view-here"); 

I also ran some tests with RedirectView, and it seems to have worked with the previous slash.

+8
source share

As soon as I used the double slash when calling the API, I got the same error.

I had to call http: // localhost: 8080 / getSomething, but I did as http: // localhost: 8080 // getSomething . I solved this by removing the extra slash.

+3
source share

The following is a clean solution to the problem. It does not compromise security, because we use the same strict firewall.

Steps to fix as shown below:

STEP 1. Create a class that overrides StrictHttpFirewall, as shown below.

 package com.biz.brains.project.security.firewall; import java.util.Arrays; import java.util.Collection; import java.util.Collections; import java.util.HashSet; import java.util.List; import java.util.Set; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import org.springframework.http.HttpMethod; import org.springframework.security.web.firewall.DefaultHttpFirewall; import org.springframework.security.web.firewall.FirewalledRequest; import org.springframework.security.web.firewall.HttpFirewall; import org.springframework.security.web.firewall.RequestRejectedException; public class CustomStrictHttpFirewall implements HttpFirewall { private static final Set<String> ALLOW_ANY_HTTP_METHOD = Collections.unmodifiableSet(Collections.emptySet()); private static final String ENCODED_PERCENT = "%25"; private static final String PERCENT = "%"; private static final List<String> FORBIDDEN_ENCODED_PERIOD = Collections.unmodifiableList(Arrays.asList("%2e", "%2E")); private static final List<String> FORBIDDEN_SEMICOLON = Collections.unmodifiableList(Arrays.asList(";", "%3b", "%3B")); private static final List<String> FORBIDDEN_FORWARDSLASH = Collections.unmodifiableList(Arrays.asList("%2f", "%2F")); private static final List<String> FORBIDDEN_BACKSLASH = Collections.unmodifiableList(Arrays.asList("\\", "%5c", "%5C")); private Set<String> encodedUrlBlacklist = new HashSet<String>(); private Set<String> decodedUrlBlacklist = new HashSet<String>(); private Set<String> allowedHttpMethods = createDefaultAllowedHttpMethods(); public CustomStrictHttpFirewall() { urlBlacklistsAddAll(FORBIDDEN_SEMICOLON); urlBlacklistsAddAll(FORBIDDEN_FORWARDSLASH); urlBlacklistsAddAll(FORBIDDEN_BACKSLASH); this.encodedUrlBlacklist.add(ENCODED_PERCENT); this.encodedUrlBlacklist.addAll(FORBIDDEN_ENCODED_PERIOD); this.decodedUrlBlacklist.add(PERCENT); } public void setUnsafeAllowAnyHttpMethod(boolean unsafeAllowAnyHttpMethod) { this.allowedHttpMethods = unsafeAllowAnyHttpMethod ? ALLOW_ANY_HTTP_METHOD : createDefaultAllowedHttpMethods(); } public void setAllowedHttpMethods(Collection<String> allowedHttpMethods) { if (allowedHttpMethods == null) { throw new IllegalArgumentException("allowedHttpMethods cannot be null"); } if (allowedHttpMethods == ALLOW_ANY_HTTP_METHOD) { this.allowedHttpMethods = ALLOW_ANY_HTTP_METHOD; } else { this.allowedHttpMethods = new HashSet<>(allowedHttpMethods); } } public void setAllowSemicolon(boolean allowSemicolon) { if (allowSemicolon) { urlBlacklistsRemoveAll(FORBIDDEN_SEMICOLON); } else { urlBlacklistsAddAll(FORBIDDEN_SEMICOLON); } } public void setAllowUrlEncodedSlash(boolean allowUrlEncodedSlash) { if (allowUrlEncodedSlash) { urlBlacklistsRemoveAll(FORBIDDEN_FORWARDSLASH); } else { urlBlacklistsAddAll(FORBIDDEN_FORWARDSLASH); } } public void setAllowUrlEncodedPeriod(boolean allowUrlEncodedPeriod) { if (allowUrlEncodedPeriod) { this.encodedUrlBlacklist.removeAll(FORBIDDEN_ENCODED_PERIOD); } else { this.encodedUrlBlacklist.addAll(FORBIDDEN_ENCODED_PERIOD); } } public void setAllowBackSlash(boolean allowBackSlash) { if (allowBackSlash) { urlBlacklistsRemoveAll(FORBIDDEN_BACKSLASH); } else { urlBlacklistsAddAll(FORBIDDEN_BACKSLASH); } } public void setAllowUrlEncodedPercent(boolean allowUrlEncodedPercent) { if (allowUrlEncodedPercent) { this.encodedUrlBlacklist.remove(ENCODED_PERCENT); this.decodedUrlBlacklist.remove(PERCENT); } else { this.encodedUrlBlacklist.add(ENCODED_PERCENT); this.decodedUrlBlacklist.add(PERCENT); } } private void urlBlacklistsAddAll(Collection<String> values) { this.encodedUrlBlacklist.addAll(values); this.decodedUrlBlacklist.addAll(values); } private void urlBlacklistsRemoveAll(Collection<String> values) { this.encodedUrlBlacklist.removeAll(values); this.decodedUrlBlacklist.removeAll(values); } @Override public FirewalledRequest getFirewalledRequest(HttpServletRequest request) throws RequestRejectedException { rejectForbiddenHttpMethod(request); rejectedBlacklistedUrls(request); if (!isNormalized(request)) { request.setAttribute("isNormalized", new RequestRejectedException("The request was rejected because the URL was not normalized.")); } String requestUri = request.getRequestURI(); if (!containsOnlyPrintableAsciiCharacters(requestUri)) { request.setAttribute("isNormalized", new RequestRejectedException("The requestURI was rejected because it can only contain printable ASCII characters.")); } return new FirewalledRequest(request) { @Override public void reset() { } }; } private void rejectForbiddenHttpMethod(HttpServletRequest request) { if (this.allowedHttpMethods == ALLOW_ANY_HTTP_METHOD) { return; } if (!this.allowedHttpMethods.contains(request.getMethod())) { request.setAttribute("isNormalized", new RequestRejectedException("The request was rejected because the HTTP method \"" + request.getMethod() + "\" was not included within the whitelist " + this.allowedHttpMethods)); } } private void rejectedBlacklistedUrls(HttpServletRequest request) { for (String forbidden : this.encodedUrlBlacklist) { if (encodedUrlContains(request, forbidden)) { request.setAttribute("isNormalized", new RequestRejectedException("The request was rejected because the URL contained a potentially malicious String \"" + forbidden + "\"")); } } for (String forbidden : this.decodedUrlBlacklist) { if (decodedUrlContains(request, forbidden)) { request.setAttribute("isNormalized", new RequestRejectedException("The request was rejected because the URL contained a potentially malicious String \"" + forbidden + "\"")); } } } @Override public HttpServletResponse getFirewalledResponse(HttpServletResponse response) { return new FirewalledResponse(response); } private static Set<String> createDefaultAllowedHttpMethods() { Set<String> result = new HashSet<>(); result.add(HttpMethod.DELETE.name()); result.add(HttpMethod.GET.name()); result.add(HttpMethod.HEAD.name()); result.add(HttpMethod.OPTIONS.name()); result.add(HttpMethod.PATCH.name()); result.add(HttpMethod.POST.name()); result.add(HttpMethod.PUT.name()); return result; } private static boolean isNormalized(HttpServletRequest request) { if (!isNormalized(request.getRequestURI())) { return false; } if (!isNormalized(request.getContextPath())) { return false; } if (!isNormalized(request.getServletPath())) { return false; } if (!isNormalized(request.getPathInfo())) { return false; } return true; } private static boolean encodedUrlContains(HttpServletRequest request, String value) { if (valueContains(request.getContextPath(), value)) { return true; } return valueContains(request.getRequestURI(), value); } private static boolean decodedUrlContains(HttpServletRequest request, String value) { if (valueContains(request.getServletPath(), value)) { return true; } if (valueContains(request.getPathInfo(), value)) { return true; } return false; } private static boolean containsOnlyPrintableAsciiCharacters(String uri) { int length = uri.length(); for (int i = 0; i < length; i++) { char c = uri.charAt(i); if (c < '\u0020' || c > '\u007e') { return false; } } return true; } private static boolean valueContains(String value, String contains) { return value != null && value.contains(contains); } private static boolean isNormalized(String path) { if (path == null) { return true; } if (path.indexOf("//") > -1) { return false; } for (int j = path.length(); j > 0;) { int i = path.lastIndexOf('/', j - 1); int gap = j - i; if (gap == 2 && path.charAt(i + 1) == '.') { // ".", "/./" or "/." return false; } else if (gap == 3 && path.charAt(i + 1) == '.' && path.charAt(i + 2) == '.') { return false; } j = i; } return true; } } 

STEP 2. Creating the FirewalledResponse Class

 package com.biz.brains.project.security.firewall; import java.io.IOException; import java.util.regex.Pattern; import javax.servlet.http.Cookie; import javax.servlet.http.HttpServletResponse; import javax.servlet.http.HttpServletResponseWrapper; class FirewalledResponse extends HttpServletResponseWrapper { private static final Pattern CR_OR_LF = Pattern.compile("\\r|\\n"); private static final String LOCATION_HEADER = "Location"; private static final String SET_COOKIE_HEADER = "Set-Cookie"; public FirewalledResponse(HttpServletResponse response) { super(response); } @Override public void sendRedirect(String location) throws IOException { // TODO: implement pluggable validation, instead of simple blacklisting. // SEC-1790. Prevent redirects containing CRLF validateCrlf(LOCATION_HEADER, location); super.sendRedirect(location); } @Override public void setHeader(String name, String value) { validateCrlf(name, value); super.setHeader(name, value); } @Override public void addHeader(String name, String value) { validateCrlf(name, value); super.addHeader(name, value); } @Override public void addCookie(Cookie cookie) { if (cookie != null) { validateCrlf(SET_COOKIE_HEADER, cookie.getName()); validateCrlf(SET_COOKIE_HEADER, cookie.getValue()); validateCrlf(SET_COOKIE_HEADER, cookie.getPath()); validateCrlf(SET_COOKIE_HEADER, cookie.getDomain()); validateCrlf(SET_COOKIE_HEADER, cookie.getComment()); } super.addCookie(cookie); } void validateCrlf(String name, String value) { if (hasCrlf(name) || hasCrlf(value)) { throw new IllegalArgumentException( "Invalid characters (CR/LF) in header " + name); } } private boolean hasCrlf(String value) { return value != null && CR_OR_LF.matcher(value).find(); } } 

STEP 3. Creating a custom filter to suppress a RejectedException

 package com.biz.brains.project.security.filter; import java.io.IOException; import java.util.Objects; import javax.servlet.FilterChain; import javax.servlet.ServletException; import javax.servlet.ServletRequest; import javax.servlet.ServletResponse; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import org.springframework.core.Ordered; import org.springframework.core.annotation.Order; import org.springframework.http.HttpHeaders; import org.springframework.security.web.firewall.RequestRejectedException; import org.springframework.stereotype.Component; import org.springframework.web.filter.GenericFilterBean; import lombok.extern.slf4j.Slf4j; @Component @Slf4j @Order(Ordered.HIGHEST_PRECEDENCE) public class RequestRejectedExceptionFilter extends GenericFilterBean { @Override public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException { try { RequestRejectedException requestRejectedException=(RequestRejectedException) servletRequest.getAttribute("isNormalized"); if(Objects.nonNull(requestRejectedException)) { throw requestRejectedException; }else { filterChain.doFilter(servletRequest, servletResponse); } } catch (RequestRejectedException requestRejectedException) { HttpServletRequest httpServletRequest = (HttpServletRequest) servletRequest; HttpServletResponse httpServletResponse = (HttpServletResponse) servletResponse; log .error( "request_rejected: remote={}, user_agent={}, request_url={}", httpServletRequest.getRemoteHost(), httpServletRequest.getHeader(HttpHeaders.USER_AGENT), httpServletRequest.getRequestURL(), requestRejectedException ); httpServletResponse.sendError(HttpServletResponse.SC_NOT_FOUND); } } } 

STEP 4: Add a custom filter to the spring filter chain in the security configuration

 @Override protected void configure(HttpSecurity http) throws Exception { http.addFilterBefore(new RequestRejectedExceptionFilter(), ChannelProcessingFilter.class); } 

Now, using the above fix, we can handle RequestRejectedException with 404 error page.

+2
source share

In my case, the problem was that you were not logged in with Postman, so I opened a connection on a different tab with the session cookie, which I took from the headers in my Chrome session.

0
source share

In my case, when upgrading from spring-securiy-web 3.1.3 to 4.2.12, the value of defaultHttpFirewall was changed from DefaultHttpFirewall to StrictHttpFirewall by default. So just define it in the XML configuration as shown below:

 <bean id="defaultHttpFirewall" class="org.springframework.security.web.firewall.DefaultHttpFirewall"/> <sec:http-firewall ref="defaultHttpFirewall"/> 

set HTTPFirewall as DefaultHttpFirewall

0
source share

All Articles