Skip to content

Commit ccd8be0

Browse files
Jamesclaude
authored andcommitted
Fix demo bugs and doc gaps from issues #40-45
- #40: document/use -Dexec.classpathScope=compile, not runtime; system-scoped deps (simple-xml, fastjson, ...) are excluded from Maven's runtime scope, breaking any demo that touches one (e.g. dlineageBasic). - #41: note in gettablecolumns/readme.md that runGetTableColumn needs a live DB connection and doesn't compile here; point at getResultColumn instead. - #42: traceColumn's main() called runFile() against a hardcoded, nonexistent Windows path with the working runText() call commented out, so it always exited 0 with zero output. Uncommented runText(); added a readme.md. - #43: traceDataLineageTest's two methods were fully commented out, testing nothing, against fixture files that don't exist in this repo. Replaced with one real test. While building it, found the actual reason the tracedatalineage demo never produces output: TGSqlParser.setSqlInputStream() silently parses to zero statements against the public trial artifact. Switched to reading the stream into sqltext, which parses correctly; added a sample/ directory and readme.md now that the demo actually works. - #44: sqlrefactor's rmdupParenthesis blocked on an undocumented interactive stdin prompt for dialect selection. Replaced it with a /t <vendor> flag, consistent with every other demo; added a readme.md. - #45: README's `head -1` trick for finding a demo's package line returns nothing when a blank line or comment precedes it; switched to `grep -m1 '^package'`. Test count in README/CLAUDE.md drops from 153 to 152 (net -1 from replacing traceDataLineageTest's two no-op methods with one real one); still 3 known analyzespTest failures, 149 (was 150) passing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012qPRpoD8exYRrUmbfXXWXj
1 parent b0fff3f commit ccd8be0

12 files changed

Lines changed: 245 additions & 95 deletions

File tree

.github/workflows/build.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,6 @@ jobs:
8686
printf 'SELECT a.id, b.name FROM ta a JOIN tb b ON a.id = b.id WHERE a.x > 1;\n' > q.sql
8787
out=$(mvn -q exec:java \
8888
-Dexec.mainClass=gudusoft.gsqlparser.demos.checksyntax.checksyntax \
89-
-Dexec.args="/f q.sql /t oracle" -Dexec.classpathScope=runtime)
89+
-Dexec.args="/f q.sql /t oracle" -Dexec.classpathScope=compile)
9090
echo "$out"
9191
grep -q "syntax errors: 0" <<<"$out"

README.md

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ SELECT a.id, b.name FROM ta a JOIN tb b ON a.id = b.id WHERE a.x > 1;
3737
SQL
3838

3939
mvn -q exec:java -Dexec.mainClass=gudusoft.gsqlparser.demos.checksyntax.checksyntax \
40-
-Dexec.args="/f q.sql /t oracle" -Dexec.classpathScope=runtime
40+
-Dexec.args="/f q.sql /t oracle" -Dexec.classpathScope=compile
4141
```
4242

4343
```text
@@ -50,7 +50,7 @@ Reformat it:
5050

5151
```bash
5252
mvn -q exec:java -Dexec.mainClass=demos.formatsql.formatsql \
53-
-Dexec.args="q.sql" -Dexec.classpathScope=runtime
53+
-Dexec.args="q.sql" -Dexec.classpathScope=compile
5454
```
5555

5656
```text
@@ -66,16 +66,26 @@ Argument conventions differ between demos: `checksyntax` takes `/f <file>` and
6666
`/t <vendor>`, while `formatsql` takes a bare filename. Run any demo with no
6767
arguments and it prints its own usage line.
6868

69+
> **Use `-Dexec.classpathScope=compile`, not `runtime`.** `pom.xml` declares
70+
> `simple-xml`, `fastjson` and a few other jars under `lib/` with
71+
> `<scope>system</scope>`, since they have no public Maven coordinate. Maven's
72+
> `runtime` classpath scope excludes `system`-scoped dependencies by design, so
73+
> any demo that touches one of them (e.g. `dlineageBasic`, which uses
74+
> `org.simpleframework.xml`) fails with `NoClassDefFoundError` under `runtime`
75+
> even though the jar is right there in `lib/`. `compile` scope includes them
76+
> and works for every demo.
77+
6978
> **Package names are not uniform yet.** A move of the demos from `demos.*` to
7079
> `gudusoft.gsqlparser.demos.*` is partly done: 176 files sit under
7180
> `src/main/java/gudusoft/` while still declaring `package demos.*`, so the class
7281
> you pass to `-Dexec.mainClass` follows the **package declaration**, not the
7382
> directory. `checksyntax` is `gudusoft.gsqlparser.demos.checksyntax.checksyntax`;
74-
> `formatsql` is `demos.formatsql.formatsql`. When in doubt, read the first line
75-
> of the source:
83+
> `formatsql` is `demos.formatsql.formatsql`. When in doubt, grep the first
84+
> `package` line — some of these files lead with a blank line or a comment, so
85+
> plain `head -1` sometimes returns nothing:
7686
>
7787
> ```bash
78-
> head -1 src/main/java/gudusoft/gsqlparser/demos/<demo>/<Demo>.java
88+
> grep -m1 '^package' src/main/java/gudusoft/gsqlparser/demos/<demo>/<Demo>.java
7989
> ```
8090
8191
## Where the parser comes from
@@ -134,7 +144,7 @@ mvn -Plocal -Dgsp.core.version=4.1.5.9 compile
134144
135145
mvn -q -Plocal -Dgsp.core.version=4.1.5.9 exec:java \
136146
-Dexec.mainClass=gudusoft.gsqlparser.demos.checksyntax.checksyntax \
137-
-Dexec.args="/f q.sql /t oracle" -Dexec.classpathScope=runtime
147+
-Dexec.args="/f q.sql /t oracle" -Dexec.classpathScope=compile
138148
```
139149
140150
This replaces the loop that existed while the demos were a vendored module of
@@ -171,14 +181,20 @@ directories carry their own `readme.md`.
171181
mvn test
172182
```
173183
174-
153 tests run. **Three currently fail**, all in
184+
152 tests run. **Three currently fail**, all in
175185
`gudusoft.gsqlparser.demosTest.analyzespTest` (`testSample1`, `testSample6`,
176186
`testSample8`). They compare stored-procedure relation output against golden
177187
strings written for an older parser build, and that output has since changed.
178188
They are left in place rather than deleted or rewritten, because they are a
179-
real signal about output drift rather than a broken harness. The other 150
189+
real signal about output drift rather than a broken harness. The other 149
180190
pass. That is why the getting-started step above uses `-DskipTests`.
181191
192+
(`traceDataLineageTest` used to contribute 2 of those "passing" tests with
193+
every line commented out — no assertions, testing nothing, referencing fixture
194+
files that never shipped with this repo. It's been replaced with one real
195+
test against inline SQL; see
196+
[#43](https://github.com/sqlparser/gsp_demo_java/issues/43).)
197+
182198
## What is excluded from the build
183199
184200
Some demos read metadata straight out of a running database over JDBC, using

src/main/java/gudusoft/gsqlparser/demos/gettablecolumns/readme.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,24 @@ For more detailed information about how this tools works, please check [this art
88
## Usage
99
`java runGetTableColumn [/f <path_to_sql_file>] [/t <database type>] [/<show option>]`
1010

11+
> **`runGetTableColumn` does not compile in this repository.** It connects to a
12+
> live database over JDBC to resolve ambiguous columns (see "Resolve the
13+
> ambiguous columns in SQL query" below), using a `TSQLDataSource` class the
14+
> public trial parser artifact doesn't ship — `pom.xml` excludes it from the
15+
> build for exactly this reason. It's only runnable from the standalone
16+
> "Binary version" below, which bundles the full metadata layer.
17+
>
18+
> For column/table extraction without a database connection, in this
19+
> repository, use `getResultColumn` instead:
20+
> ```bash
21+
> mvn -q exec:java -Dexec.mainClass=demos.gettablecolumns.getResultColumn -Dexec.classpathScope=compile
22+
> ```
23+
> It parses an inline query and prints its result columns; edit the `sqltext`
24+
> in `getResultColumn.java` to try your own SQL. Other classes in this folder
25+
> (`columnTableStmt`, `columnsInResultColumn`, `whatClause`,
26+
> `tableColumnRename`) demonstrate related, database-free column/table
27+
> analysis and build the same way.
28+
1129
## Binary version
1230
https://www.gudusoft.com/gsp_java/gettablecolumn.zip
1331
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
## Description
2+
3+
`rmdupParenthesis` removes redundant duplicated parentheses from a SQL
4+
statement using the source token list, e.g. `WHERE ((a.x > 1))` becomes
5+
`WHERE (a.x > 1)`.
6+
7+
## Usage
8+
9+
```
10+
java rmdupParenthesis <sqlfile.sql> [/t <database type>]
11+
```
12+
13+
`/t` is optional and defaults to `oracle`.
14+
15+
```bash
16+
cat > paren.sql <<'SQL'
17+
SELECT * FROM ta WHERE ((a.x > 1));
18+
SQL
19+
20+
mvn -q exec:java -Dexec.mainClass=demos.sqlrefactor.rmdupParenthesis \
21+
-Dexec.args="paren.sql /t mssql" -Dexec.classpathScope=runtime
22+
```
23+
24+
```
25+
Selected SQL dialect: dbvmssql
26+
SELECT * FROM ta WHERE (a.x > 1);
27+
Time Escaped: 764
28+
```

src/main/java/gudusoft/gsqlparser/demos/sqlrefactor/rmdupParenthesis.java

Lines changed: 9 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,9 @@
55

66
import gudusoft.gsqlparser.*;
77

8-
import java.io.BufferedReader;
98
import java.io.File;
10-
import java.io.IOException;
11-
import java.io.InputStreamReader;
9+
import java.util.Arrays;
10+
import java.util.List;
1211

1312
public class rmdupParenthesis {
1413

@@ -46,8 +45,10 @@ public static void main(String args[])
4645
long t;
4746
t = System.currentTimeMillis();
4847

49-
if (args.length != 1){
50-
System.out.println("Usage: java rmdupParenthesis sqlfile.sql");
48+
List<String> argList = Arrays.asList(args);
49+
if (args.length < 1){
50+
System.out.println("Usage: java rmdupParenthesis <sqlfile.sql> [/t <database type>]");
51+
System.out.println(" /t <type> - Specify database type (default: oracle)");
5152
return;
5253
}
5354
File file=new File(args[0]);
@@ -57,27 +58,9 @@ public static void main(String args[])
5758
}
5859

5960
EDbVendor dbVendor = EDbVendor.dbvoracle;
60-
String msg = "Please select SQL dialect: 1: SQL Server, 2: Oralce, 3: MySQL, 4: DB2, 5: PostGRESQL, 6: Teradta, default is 2: Oracle";
61-
System.out.println(msg);
62-
63-
BufferedReader br=new BufferedReader(new InputStreamReader(System.in));
64-
try{
65-
int db = Integer.parseInt(br.readLine());
66-
if (db == 1){
67-
dbVendor = EDbVendor.dbvmssql;
68-
}else if(db == 2){
69-
dbVendor = EDbVendor.dbvoracle;
70-
}else if(db == 3){
71-
dbVendor = EDbVendor.dbvmysql;
72-
}else if(db == 4){
73-
dbVendor = EDbVendor.dbvdb2;
74-
}else if(db == 5){
75-
dbVendor = EDbVendor.dbvpostgresql;
76-
}else if(db == 6){
77-
dbVendor = EDbVendor.dbvteradata;
78-
}
79-
}catch(IOException i) {
80-
}catch (NumberFormatException numberFormatException){
61+
int index = argList.indexOf("/t");
62+
if (index != -1 && args.length > index + 1){
63+
dbVendor = TGSqlParser.getDBVendorByName(args[index + 1]);
8164
}
8265

8366
System.out.println("Selected SQL dialect: "+dbVendor.toString());
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
## Description
2+
3+
Follows each result column of a `SELECT` (or the subquery of a `CREATE VIEW`)
4+
back to the base-table columns and expressions it's built from, printing an
5+
indented trace: alias, the column's own expression, then each source column it
6+
reads from, recursing into subqueries, derived tables and nested expressions.
7+
8+
## Usage
9+
10+
`runTraceColumn` runs a single query that's inline in `main()` — there's no
11+
`/f` file argument or CLI parsing here, unlike most other demos. Run it with:
12+
13+
```bash
14+
mvn -q exec:java -Dexec.mainClass=demos.traceColumn.runTraceColumn -Dexec.classpathScope=compile
15+
```
16+
17+
To trace a different query, edit the `sqltext` (or the vendor passed to
18+
`runText`) in `runTraceColumn.java` and rebuild, or call
19+
`TTraceColumn` directly from your own code:
20+
21+
```java
22+
TTraceColumn traceColumn = new TTraceColumn(EDbVendor.dbvoracle);
23+
traceColumn.runText("SELECT a.id, b.name FROM ta a JOIN tb b ON a.id = b.id");
24+
System.out.print(traceColumn.getInfos().toString());
25+
```
26+
27+
## Sample output
28+
29+
For the query built into `runTraceColumn.java`:
30+
31+
```
32+
"Department"
33+
-->a.deptno(expr)
34+
-->a.deptno
35+
-->deptno(expr)
36+
-->scott.emp.deptno
37+
"Employees"
38+
-->a.num_emp/b.total_count(expr)
39+
-->a.num_emp
40+
-->COUNT(*)(expr)
41+
-->scott.emp.*
42+
-->b.total_count
43+
-->COUNT(*)(expr)
44+
-->scott.emp.*
45+
...
46+
```
47+
48+
Each `-->` level is one hop back through the expression tree; a line ending in
49+
`(expr)` is the expression at that point, and the line above it (when present)
50+
is the alias or source column it was reached through.

src/main/java/gudusoft/gsqlparser/demos/traceColumn/runTraceColumn.java

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,7 @@ public static void main(String args[]) {
2727
// "from table1 a join table2 b on a.id=b.id\n" +
2828
// "join table3 c on b.tid=c.id";
2929

30-
// runText(EDbVendor.dbvoracle,sqltext);
31-
runFile(EDbVendor.dbvoracle,"c:/prg/tmp/demo.sql");
30+
runText(EDbVendor.dbvoracle,sqltext);
3231
}
3332

3433
public static void runText(EDbVendor dbVendor, String query){
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
## Description
2+
3+
Reads every `.sql` file in a directory (T-SQL / `dbvmssql` syntax) as one
4+
combined script, builds up a model of the tables it creates and the
5+
`INSERT ... SELECT` / `UPDATE` statements and stored procedures that move data
6+
between them, then traces each base-table column forward to every column it
7+
ultimately feeds into. This is a directory/multi-script tool, unlike most
8+
other demos here which parse one query or file — see `traceColumn` for
9+
tracing a single query's result columns back to their sources instead.
10+
11+
## Usage
12+
13+
```
14+
java demos.tracedatalineage.traceDataLineage <sql scripts directory path> [<output file path>]
15+
```
16+
17+
The first argument **must be a directory**, not a single `.sql` file —
18+
passing a file silently matches nothing (`SqlFileList` only lists `.sql` files
19+
inside a directory) and the tool exits 0 with no output, which looks
20+
identical to "ran fine, found no lineage." The optional second argument
21+
writes the result to a file instead of stdout.
22+
23+
```bash
24+
mvn -q exec:java -Dexec.mainClass=demos.tracedatalineage.traceDataLineage \
25+
-Dexec.args="src/main/java/gudusoft/gsqlparser/demos/tracedatalineage/sample" \
26+
-Dexec.classpathScope=compile
27+
```
28+
29+
```
30+
source_tbl.id -----> target_tbl.id
31+
source_tbl.amount -----> target_tbl.total
32+
```
33+
34+
`sample/` contains the two-file script this example reads: one file creates
35+
`source_tbl` and `target_tbl`, the other loads the first into the second with
36+
`INSERT INTO target_tbl (id, total) SELECT id, amount FROM source_tbl`.
37+
38+
Lineage is only recorded across `INSERT`/`UPDATE` statements and stored
39+
procedures — a `CREATE VIEW` that merely selects from a table does not, on its
40+
own, produce a traceable relation here.
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
CREATE TABLE source_tbl (
2+
id INT,
3+
amount INT
4+
);
5+
6+
CREATE TABLE target_tbl (
7+
id INT,
8+
total INT
9+
);
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
INSERT INTO target_tbl (id, total)
2+
SELECT id, amount FROM source_tbl;

0 commit comments

Comments
 (0)