Comments from original TOML end up in the wrong spot

[jcote-tc]

Hello,

I am using the tomlkit Python library to modify an existing TOML file.

The original file looks like this:

## My POST route comment
[[routes]]
method = 'POST'
path = '/my/post/path'
topic = 'my-topic.0'

## My GET route comment
[[routes]]
method = 'GET'
path = '/my/get/path/{key}'
topic = 'my-topic.1'

I add two tables to that original array of tables "[[routes]]", with method = 'POST' and path = '/my/post/path'. When I do that, the resulting file becomes:

## My POST route comment
[[routes]]
method = 'POST'
path = '/my/post/path'
topic = 'my-topic.0'

## My GET route comment

[routes.rate_limit]
enabled = true

[routes.rate_limit.requests]
per_second = 25

[[routes]]
method = 'GET'
path = '/my/get/path/{key}'
topic = 'my-topic.1'

The comment ends up in the wrong spot. What can I do to make sure it stays on top of my second "[[routes]]" such as:

## My POST route comment
[[routes]]
method = 'POST'
path = '/my/post/path'
topic = 'my-topic.0'

[routes.rate_limit]
enabled = true

[routes.rate_limit.requests]
per_second = 25

## My GET route comment
[[routes]]
method = 'GET'
path = '/my/get/path/{key}'
topic = 'my-topic.1'

This is an extract of the code I use to add my 2 tables:

# result is <class 'tomlkit.items.Table'>
route_ratelimit = tomlkit.table()
route_ratelimit.add("enabled", True)
route_ratelimit_requests = tomlkit.table()
route_ratelimit_requests.add("per_second", 25)
result.append("rate_limit", route_ratelimit)
route_ratelimit.append("requests", route_ratelimit_requests)

Thank you for your assistance.

[frostming]

It's hard for such level of style-preserving. The parser couldn't know what table the comments should adhere to.

[sallner]

Would it be an option to add the comment inside the [[routes]] sections/entries?

[jcote-tc]

Would it be an option to add the comment inside the [[routes]] sections/entries?

I am not in control of the original file - but all comments are on top of the sections so it makes it easier to handle that use case. I wrote a python function to handle that scenario. Basically I find the index/location where I need to insert my new Table, check if the line above is a Comment and if so I do -1 to the index, which preserves the location of the commented line:

def search_for_insertion_point_index(parsed_toml):
    key_counter = 0
    stored_keys = []
    for k in parsed_toml.body:
        stored_keys.append(k)
        key_counter += 1
        if isinstance(k[1], tomlkit.items.Table):
            # insert new table before the first found tomlkit.items.Table
            # check if the last line is a comment
            if isinstance(stored_keys[parsed_toml.body.index(k) - 1][1], tomlkit.items.Comment):
                #print("Comment found on the line before. Return index minus 1!")
                return parsed_toml.body.index(k) - 1
            #print(parsed_toml.body.index(k))
            return parsed_toml.body.index(k)
        if isinstance(k[1], tomlkit.items.AoT):
            # insert new table before the first found tomlkit.items.AoT
            # check if the line before is a comment
            if isinstance(stored_keys[parsed_toml.body.index(k) - 1][1], tomlkit.items.Comment):
                #print("Comment found on the line before. Return index minus 1!")
                return parsed_toml.body.index(k) - 1
            #print(parsed_toml.body.index(k))
            return parsed_toml.body.index(k)

[gronerl]

I am facing the same issue. I agree that there is no general way to possibly know what the comments semantically belong with, but I think the current defaults could be improved on.

Example 1:
Consider this:

[table]
material = "wood"

# [chair]
# material = "metal"

[lamp]
 = "value"

The current default is to insert key/value pairs as the last line before the next table. If I insert table.extendable = true this will generate:

[table]
material = "wood"

# [chair]
# material = "metal"

table.extendable = true
[lamp]
type = "rgb

Which will be wrong if the previosuly commented table is uncommented.

Example 2:
Consider this:

[table]
material = "wood"

# this table describes lamps
[lamp]

type = "rgb

Insert table.extendable = true:

[table]
material = "wood"

# this table describes lamps
table.extendable = true
[lamp]

type = "rgb"

As in the original issue.

In general, I think comments rarely ever refer to what came before.
My suggestion would be to modify the behavior to: Insert new table entries before all trailing comments or whitespace of the current table.

I can think of one exception to this observation, namely:

[table]
# this section holds data of a table
material = "wood"

Note that the proposed rule continues to give sane results here. The only case that will look awkward is if we insert into a table that is commented in this style but otherwise empty.