Skip to content

Http oracle timeout#1608

Open
omursahin wants to merge 8 commits into
masterfrom
http-oracle-timeout
Open

Http oracle timeout#1608
omursahin wants to merge 8 commits into
masterfrom
http-oracle-timeout

Conversation

@omursahin

@omursahin omursahin commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Python test case

    # Calls:
    # (null) GET:/api/timeout/slow/{id}
    # Found 1 potential fault of type-code 921
    @timeout_decorator.timeout(60)
    def test_0_get_on_slow_timeout(self):
        
        # Fault921. Timeout.
        headers = {}
        headers['Accept'] = "*/*"
        with self.assertRaises(Exception):
            requests \
                    .get(self.baseUrlOfSut + "/api/timeout/slow/683",
                        headers=headers, timeout=EM_HTTP_TIMEOUT, verify=False)

JS test case

/**
* Calls:
* (null) GET:/api/timeout/slow/{id}
* Found 1 potential fault of type-code 921
*/
test("test_0_GetOnSlowTimeout", async () => {
    
    // Fault921. Timeout.
    await expect(superagent
            .get(baseUrlOfSut + "/api/timeout/slow/683")
            .timeout({response: EM_HTTP_TIMEOUT_MS, deadline: EM_HTTP_TIMEOUT_MS}).set('Accept', "*/*")
            .ok(res => res.status)).rejects.toThrow();
});

Kotlin:

        @BeforeAll
        @JvmStatic
        fun initClass() {
            RestAssured.enableLoggingOfRequestAndResponseIfValidationFails()
            RestAssured.useRelaxedHTTPSValidation()
            RestAssured.urlEncodingEnabled = false
            RestAssured.config = RestAssured.config()
                .jsonConfig(JsonConfig.jsonConfig().numberReturnType(JsonPathConfig.NumberReturnType.DOUBLE))
                .redirect(redirectConfig().followRedirects(false))
                .httpClient(HttpClientConfig.httpClientConfig()
                    .setParam("http.socket.timeout", 2000)
                    .setParam("http.connection.timeout", 2000))
                .jsonConfig(JsonConfig.jsonConfig().numberReturnType(JsonPathConfig.NumberReturnType.DOUBLE))
                .redirect(redirectConfig().followRedirects(false))
                .encoderConfig(EncoderConfig.encoderConfig().encodeContentTypeAs("application/octet-stream", ContentType.TEXT))
        }

    /**
    * Calls:
    * (null) GET:/api/timeout/slow/{id}
    * Found 1 potential fault of type-code 921
    */
    @Test @Timeout(60)
    fun test_0_getOnSlowTimeout()  {
        
        try{
            // Fault921. Timeout.
            given().accept("*/*")
                    .get("${baseUrlOfSut}/api/timeout/slow/683")
            fail("Expected a timeout");
        } catch(e: Exception){
        }
    }

Java

    @BeforeClass
    public static void initClass() {
        RestAssured.enableLoggingOfRequestAndResponseIfValidationFails();
        RestAssured.useRelaxedHTTPSValidation();
        RestAssured.urlEncodingEnabled = false;
        RestAssured.config = RestAssured.config()
            .jsonConfig(JsonConfig.jsonConfig().numberReturnType(JsonPathConfig.NumberReturnType.DOUBLE))
            .redirect(redirectConfig().followRedirects(false))
            .httpClient(HttpClientConfig.httpClientConfig()
                .setParam("http.socket.timeout", 2000)
                .setParam("http.connection.timeout", 2000))
            .jsonConfig(JsonConfig.jsonConfig().numberReturnType(JsonPathConfig.NumberReturnType.DOUBLE))
            .redirect(redirectConfig().followRedirects(false))
            .encoderConfig(EncoderConfig.encoderConfig().encodeContentTypeAs("application/octet-stream", ContentType.TEXT));
    }
        /**
    * Calls:
    * (null) GET:/api/timeout/slow/{id}
    * Found 1 potential fault of type-code 921
    */
    @Test(timeout = 60000)
    public void test_0_getOnSlowTimeout() throws Exception {
        
        try{
            // Fault921. Timeout.
            given().accept("*/*")
                    .get(baseUrlOfSut + "/api/timeout/slow/683");
            fail("Expected a timeout");
        } catch(Exception e){
        }
    }

@omursahin omursahin requested a review from arcuri82 July 1, 2026 10:51
//in SuperAgent, verb must be first
handleVerbEndpoint(baseUrlOfSut, call, lines)
//client timeout, same source as fuzzing tcpTimeoutMs
lines.add(".timeout({response: EM_HTTP_TIMEOUT_MS, deadline: EM_HTTP_TIMEOUT_MS})")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this EM_HTTP_TIMEOUT_MS is referring to a variable declared elsewhere? in such case, the name of the variable should be put in a shared constant


// message for the assertion that flags a missing expected timeout (Java/Kotlin/C#)
// JS uses await expect(...).rejects.toThrow() and Python uses with self.assertRaises(...)
private const val EXPECTED_TIMEOUT_MSG = "Expected a timeout"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use JavaDoc style /** */ for comments on fields and methos

lines.add("process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0';")
lines.add("const superagent = require(\"superagent\");")
// HTTP client timeout (ms)
lines.add("const EM_HTTP_TIMEOUT_MS = ${config.tcpTimeoutMs};")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this name EM_HTTP_TIMEOUT_MS should be in a shared constant, as re-used in few places

}
lines.add("from $pythonUtilsFilenameNoExtension import *")
// HTTP client timeout (seconds)
lines.add("EM_HTTP_TIMEOUT = ${config.tcpTimeoutMs / 1000.0}")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shared constant

analyzeHttpSemantics(individual, actionResults, fv)
}

if(config.blackBox && config.isEnabledFaultCategory(ExperimentalFaultCategory.HTTP_TIMEOUT)){

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comment specifying why we are skipping for white-box testing

add(".get(self.baseUrlOfSut + \"/\",")
indent()
add("headers=headers, verify=False)")
add("headers=headers, timeout=EM_HTTP_TIMEOUT, verify=False)")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

constant

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise, otherwise if we change name in future, we will break lot of places

res_0 = requests \
.get(self.baseUrlOfSut + "/foo",
headers=headers, verify=False)
headers=headers, timeout=EM_HTTP_TIMEOUT, verify=False)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

constant. also, we need a way to make it easier and do not confuse when we us MS or S version

) { args: MutableList<String> ->

setOption(args, "useExperimentalOracles", "true")
setOption(args, "tcpTimeoutMs", "2000")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this test fail because by default we have httpOracles being false?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants