There is an idea to optimize the code for json.lua

[DnsIs]

When used .. in loops, the lua will create a lot of garbage and thus affect performance. I changed your script (json-fix.lua)

c:\test>busybox diff -U 0 json.lua json-fix.lua
--- json.lua
+++ json-fix.lua
@@ -41 +41 @@
-local json = { _version = "0.1.2" }
+local json = { _version = "0.1.2fix" }
@@ -235 +235 @@
-  local res = ""
+  local res = {}
@@ -246 +246 @@
-      res = res .. str:sub(k, j - 1)
+      table.insert(res, str:sub(k, j - 1))
@@ -253 +253 @@
-        res = res .. parse_unicode_escape(hex)
+        table.insert(res, parse_unicode_escape(hex))
@@ -259 +259 @@
-        res = res .. escape_char_map_inv[c]
+        table.insert(res, escape_char_map_inv[c])
@@ -264,2 +264,2 @@
-      res = res .. str:sub(k, j - 1)
-      return res, j + 1
+      table.insert(res, str:sub(k, j - 1))
+      return table.concat(res), j + 1

I created a .HAR file (size 313971312 bytes) using firefox and tested it with a simple Lua script.

local function read(file)
	local f, err = io.open(file, "r");
	if not f then
		return;
	else
		local data = f:read("a");
		f:close();
		return (data);
	end
end

local function test(obj)
	print (obj._version)
	local last = os.time();
	local har = obj.decode(read(file));
	print ('decode - ' .. os.time() - last .. " sec")
end

json    = require "json";
jsonfix = require "json-fix";
file    = 'Archive [21-05-25 11-43-43].har';

test(json)
test(jsonfix)

I got results like this on my computer (Intel(R) Core(TM) i3-9100 CPU @ 3.60GHz)

c:\test>lua read_har.lua
0.1.2
decode - 145 sec
0.1.2fix
decode - 23 sec

c:\test>lua read_har.lua
0.1.2
decode - 144 sec
0.1.2fix
decode - 24 sec

c:\test>lua read_har.lua
0.1.2
decode - 147 sec
0.1.2fix
decode - 22 sec

[IsFaser]

(Translation)
Using "concat" is a good idea. I think it can be improved. "table.insert" can be modified to "[#tab +1] = value". The following is the test data

local tab = {}
for i=1, 10000000 do
-- table.insert(tab, i)
-- time : 8.61 7.33 7.21 s

-- tab[#tab +1] = i
-- time : 5.54 5.25 5.46 s

-- tab[table.size(tab) +1] = i
-- time : so long time
end

[DnsIs]

I completely agree with you.
Did the same test, same results.
Super.

".." - should be avoided for large amounts of data.

[appgurueu]

String concatenation using .. should never be done in loops as it incurs quadratic time complexity; each .. has to copy the entire string in linear time.

Using find + gsub could be more efficient than a rope (a rope takes 8 bytes per character, ugh), but seems to be hard to implement as the Unicode surrogate pair handling as it is currently implemented doesn't mesh well with it.

table.insert vs res[#res + 1] doesn't make much of a difference on LuaJIT, but on PUC Lua, the latter is indeed noticeably faster.

[thevurv]

If you want a fast implementation: https://github.com/Vurv78/qjson.lua

Runs thousands of times faster than this repo

LuaJIT 2.1.0-beta3
11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz

Running benchmarks...
| Name (Decode)   | Min        | Max        | Avg        | Avg / Best  |
| vurv78/qjson    | 0.008763   | 0.010837   | 0.00939495 | x1.72715    |
| rxi/json        | 0.00475    | 0.007055   | 0.00543957 | x1          |
| actboy168/json  | 0.010547   | 0.013259   | 0.0112183  | x2.06235    |
| luadist/dkjson  | 0.011222   | 0.014534   | 0.0126976  | x2.3343     |

| Name (Encode)   | Min        | Max        | Avg        | Avg / Best  |
| vurv78/qjson    | 0.001677   | 0.002637   | 0.00189174 | x1          |
| rxi/json        | 0.010513   | 0.011322   | 0.010924   | x5.77459    |
| actboy168/json  | 0.009892   | 0.012293   | 0.0104864  | x5.54327    |
| luadist/dkjson  | 0.014829   | 0.01985    | 0.0157059  | x8.30237    |

Lua 5.3
11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz

Running benchmarks...
| Name (Decode)   | Min        | Max        | Avg        | Avg / Best  |
| luadist/dkjson  | 0.028568   | 0.033705   | 0.0306484  | x1.94652    |
| rxi/json        | 0.045178   | 0.053548   | 0.0480421  | x3.05121    |
| vurv78/qjson    | 0.015006   | 0.018043   | 0.0157452  | x1          |
| actboy168/json  | 0.019061   | 0.023373   | 0.0200551  | x1.27372    |

| Name (Encode)   | Min        | Max        | Avg        | Avg / Best  |
| luadist/dkjson  | 0.021639   | 0.024754   | 0.0226422  | x4.10556    |
| rxi/json        | 0.015463   | 0.019618   | 0.0166444  | x3.01802    |
| vurv78/qjson    | 0.005148   | 0.006336   | 0.00551502 | x1          |
| actboy168/json  | 0.016263   | 0.018331   | 0.0170535  | x3.09218    |

[appgurueu]

If you want a fast implementation: https://github.com/Vurv78/qjson.lua

Runs thousands of times faster than this repo

Sorry, but that claim in that form simply isn't true (or at best highly misleading). On which inputs? Since this is a complexity issue, any linear-time implementation can be arbitrarily faster.

Also, your parser doesn't seem spec-compliant to me. I see no provisions for unicode escape sequences.
It also seems to have the same complexity issue as this one: It doesn't properly use a rope for encoding.

Your isarray relies on undocumented behavior (pairs degrading to ipairs).

Formatting strings with %q does not always produce valid JSON.

TL;DR: Your JSON parser is broken and has a time complexity issue as well. Your performance claims in their current form can't be taken seriously.

Besides, my PR fixes the complexity issue.

[yogwoggf]

If you want a fast implementation: https://github.com/Vurv78/qjson.lua
Runs thousands of times faster than this repo

Sorry, but that claim in that form simply isn't true (or at best highly misleading). On which inputs? Since this is a complexity issue, any linear-time implementation can be arbitrarily faster.

Also, your parser doesn't seem spec-compliant to me. I see no provisions for unicode escape sequences. It also seems to have the same complexity issue as this one: It doesn't properly use a rope for encoding.

Your isarray relies on undocumented behavior (pairs degrading to ipairs).

Formatting strings with %q does not always produce valid JSON.

TL;DR: Your JSON parser is broken and has a time complexity issue as well. Your performance claims in their current form can't be taken seriously.

Besides, my PR fixes the complexity issue.

There are benchmarks. It is better.

[appgurueu]

There are benchmarks. It is better.

I don't see how you are replying to any of my points.

[yogwoggf]

There are benchmarks. It is better.

I don't see how you are replying to any of my points.

I did. All of your points are invalidated by the benchmarks. Theory is theory but it comes down to the actual performance.

[appgurueu]

I did. All of your points are invalidated by the benchmarks. Theory is theory but it comes down to the actual performance.

Oh yeah? So something being fast somehow magically makes it correct? Look, I have the fastest JSON decoder:

function read_json(str) return 1 end

it's easy to optimize e.g. string decoding using gsub if you don't have to care about unicode escape sequences

"The" benchmarks of rxi/json.lua are completely insufficient; they can't claim to be representative. For the theoretical complexity issue I'm outlining, a benchmark can be trivially construed; in fact I've done it for an issue on this repo.

[appgurueu]

Just to be sure, I ran the benchmarks of rxi/json.lua against qjson.lua. Here are the results:

[decode]
Lua version   : Lua 5.4
CPU name      : AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx
../json.lua   : 0.0773s [x2.21] (min: 0.0753s, max 0.095s)
../qjson.lua  : 0.035s [x1] (min: 0.0334s, max 0.0455s)
dkjson.lua    : 0.0785s [x2.24] (min: 0.0725s, max 0.1s)
jfjson.lua    : 0.104s [x2.98] (min: 0.0992s, max 0.134s)

[encode]
Lua version   : Lua 5.4
CPU name      : AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx
../json.lua   : 0.0287s [x1.63] (min: 0.0269s, max 0.0353s)
../qjson.lua  : 0.0176s [x1] (min: 0.016s, max 0.0232s)
dkjson.lua    : 0.0385s [x2.19] (min: 0.0357s, max 0.0485s)
jfjson.lua    : 0.0507s [x2.88] (min: 0.0498s, max 0.054s)
json4lua.lua  : 0.0476s [x2.7] (min: 0.0465s, max 0.0604s)

and here's LuaJIT

[decode]
Lua version   : LuaJIT 2.1.0-beta3
CPU name      : AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx
../json.lua   : 0.0134s [x1] (min: 0.0123s, max 0.0147s)
../qjson.lua  : 0.0216s [x1.61] (min: 0.0211s, max 0.0227s)
dkjson.lua    : 0.0286s [x2.13] (min: 0.0276s, max 0.0296s)
jfjson.lua    : 0.0191s [x1.43] (min: 0.0185s, max 0.0209s)

[encode]
Lua version   : LuaJIT 2.1.0-beta3
CPU name      : AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx
../json.lua   : 0.0153s [x2.92] (min: 0.0148s, max 0.0161s)
../qjson.lua  : 0.00524s [x1] (min: 0.0047s, max 0.0056s)
dkjson.lua    : 0.0169s [x3.23] (min: 0.0163s, max 0.0182s)
jfjson.lua    : 0.017s [x3.23] (min: 0.0166s, max 0.0181s)
json4lua.lua  : 0.0229s [x4.37] (min: 0.0224s, max 0.0239s)

(oops, looks like qjson.lua isn't even always the fastest, despite omitting features and implementing JSON slightly incorrectly)

I don't know what you did wrong when benchmarking; qjson.lua is faster (about 2x), but definitely not a thousand or tens of thousands times for the given benchmarks.

Perhaps do some benchmarks before citing them?

[yogwoggf]

It's probably because you have a horribly slow CPU.