Maybe we should just trust the regex engine? I don't think the changes are necessary
Pinned
cnodejs/nodeclub
9.1k
⚡ :baby_chick:Nodeclub 是使用 Node.js 和 MongoDB 开发的社区系统
alsotang/node-lessons
1.63w
⚡ :closed_book:《Node.js 包教不包会》 by alsotang
alsotang/async_demo
585
⚡ :blue_book:async.js 各种函数的 demo
alsotang/fast-js
245
⚡ :heart_eyes: Writing Fast JavaScript
alsotang/is-chinese
97
⚡ Determine if a string is all Chinese(based on unicode range)
alsotang/cf_workers__file
6
⚡ Generate arbitrary size file on Cloudflare Workers
Activity
Aug
6
2 months ago
Aug
4
2 months ago
Aug
3
2 months ago
alsotang push alsotang/is_chinese_rs
alsotang
use is_ascii
commit sha： 9244de385b3c5b8a81bcaa4f66bf34f237d9c0c9
push time in 2 months ago
Aug
2
2 months ago
alsotang issue comment alsotang/is-chinese
alsotang
reduce range of unicode
before:
isChinese("扁担宽，板凳长，扁担想绑在板凳上。") x 11,562,495 ops/sec ±0.56% (88 runs sampled)
isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。") x 41,004,072 ops/sec ±2.48% (87 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。ss") x 29,389,279 ops/sec ±2.31% (88 runs sampled)
isChinese(chars1000) true x 332,923 ops/sec ±1.06% (89 runs sampled)
isChinese(chars1000WithS) false x 1,690,598 ops/sec ±1.35% (86 runs sampled)
Fastest is isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。")
after:
isChinese("扁担宽，板凳长，扁担想绑在板凳上。") x 11,947,991 ops/sec ±0.45% (90 runs sampled)
isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。") x 42,548,217 ops/sec ±1.18% (89 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。ss") x 30,718,944 ops/sec ±0.48% (90 runs sampled)
isChinese(chars1000) true x 375,754 ops/sec ±0.47% (90 runs sampled)
isChinese(chars1000WithS) false x 1,746,008 ops/sec ±0.89% (88 runs sampled)
Fastest is isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。")
alsotang
These ranges would be compiled to a regex. Maybe regex combines the range internally, so we don't need to do it manually?
alsotang issue comment alsotang/is-chinese
alsotang
reduce range of unicode
before:
isChinese("扁担宽，板凳长，扁担想绑在板凳上。") x 11,562,495 ops/sec ±0.56% (88 runs sampled)
isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。") x 41,004,072 ops/sec ±2.48% (87 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。ss") x 29,389,279 ops/sec ±2.31% (88 runs sampled)
isChinese(chars1000) true x 332,923 ops/sec ±1.06% (89 runs sampled)
isChinese(chars1000WithS) false x 1,690,598 ops/sec ±1.35% (86 runs sampled)
Fastest is isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。")
after:
isChinese("扁担宽，板凳长，扁担想绑在板凳上。") x 11,947,991 ops/sec ±0.45% (90 runs sampled)
isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。") x 42,548,217 ops/sec ±1.18% (89 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。ss") x 30,718,944 ops/sec ±0.48% (90 runs sampled)
isChinese(chars1000) true x 375,754 ops/sec ±0.47% (90 runs sampled)
isChinese(chars1000WithS) false x 1,746,008 ops/sec ±0.89% (88 runs sampled)
Fastest is isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。")
alsotang
I checked out this commit and compare the benchmark with master
$ node -v
v16.4.0
node benchmark.js
isChinese("扁担宽，板凳长，扁担想绑在板凳上。") x 10,532,167 ops/sec ±0.84% (87 runs sampled)
isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。") x 41,371,203 ops/sec ±0.74% (91 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。ss") x 28,886,915 ops/sec ±0.98% (88 runs sampled)
isChinese(chars1000) true x 317,638 ops/sec ±0.89% (87 runs sampled)
isChinese(chars1000WithS) false x 1,611,493 ops/sec ±0.99% (88 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。") x 10,549,005 ops/sec ±1.08% (88 runs sampled)
isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。") x 38,652,878 ops/sec ±5.66% (84 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。ss") x 28,347,738 ops/sec ±2.94% (87 runs sampled)
isChinese(chars1000) true x 299,741 ops/sec ±1.77% (83 runs sampled)
isChinese(chars1000WithS) false x 1,571,945 ops/sec ±1.43% (83 runs sampled)
There is no improvement
alsotang issue comment alsotang/is-chinese
alsotang
reduce range of unicode
before:
isChinese("扁担宽，板凳长，扁担想绑在板凳上。") x 11,562,495 ops/sec ±0.56% (88 runs sampled)
isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。") x 41,004,072 ops/sec ±2.48% (87 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。ss") x 29,389,279 ops/sec ±2.31% (88 runs sampled)
isChinese(chars1000) true x 332,923 ops/sec ±1.06% (89 runs sampled)
isChinese(chars1000WithS) false x 1,690,598 ops/sec ±1.35% (86 runs sampled)
Fastest is isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。")
after:
isChinese("扁担宽，板凳长，扁担想绑在板凳上。") x 11,947,991 ops/sec ±0.45% (90 runs sampled)
isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。") x 42,548,217 ops/sec ±1.18% (89 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。ss") x 30,718,944 ops/sec ±0.48% (90 runs sampled)
isChinese(chars1000) true x 375,754 ops/sec ±0.47% (90 runs sampled)
isChinese(chars1000WithS) false x 1,746,008 ops/sec ±0.89% (88 runs sampled)
Fastest is isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。")
alsotang
This modify is good. The overhead only happens when library initial, so it's ok. Just assure the combined result is correct and I will merge this.
alsotang issue comment alsotang/is_chinese_rs
alsotang
alsotang
You said there are some bugs in your solution, but the test cases couldn't reflect them. Can you help to add test cases?
alsotang issue comment alsotang/is_chinese_rs
alsotang
alsotang
You can create a simd branch in this repo and write it in readme. But I won't merge this into main until simd is a stable feature.
Is this ok?
alsotang issue comment alsotang/is_chinese_rs
alsotang
alsotang
This optimization can not run on stable rust??
Compiling packed_simd_2 v0.3.5
error[E0554]: `#![feature]` may not be used on the stable release channel
--> /Users/atang/.cargo/registry/src/github.com-1ecc6299db9ec823/packed_simd_2-0.3.5/src/lib.rs:214:1
|
214 | / #![feature(
215 | | const_generics,
216 | | repr_simd,
217 | | rustc_attrs,
... |
227 | | llvm_asm
228 | | )]
| |__^
error: aborting due to previous error
For more information about this error, try `rustc --explain E0554`.
error: could not compile `packed_simd_2`
alsotang issue comment alsotang/is-chinese
alsotang
reduce range of unicode
before:
isChinese("扁担宽，板凳长，扁担想绑在板凳上。") x 11,562,495 ops/sec ±0.56% (88 runs sampled)
isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。") x 41,004,072 ops/sec ±2.48% (87 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。ss") x 29,389,279 ops/sec ±2.31% (88 runs sampled)
isChinese(chars1000) true x 332,923 ops/sec ±1.06% (89 runs sampled)
isChinese(chars1000WithS) false x 1,690,598 ops/sec ±1.35% (86 runs sampled)
Fastest is isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。")
after:
isChinese("扁担宽，板凳长，扁担想绑在板凳上。") x 11,947,991 ops/sec ±0.45% (90 runs sampled)
isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。") x 42,548,217 ops/sec ±1.18% (89 runs sampled)
isChinese("扁担宽，板凳长，扁担想绑在板凳上。ss") x 30,718,944 ops/sec ±0.48% (90 runs sampled)
isChinese(chars1000) true x 375,754 ops/sec ±0.47% (90 runs sampled)
isChinese(chars1000WithS) false x 1,746,008 ops/sec ±0.89% (88 runs sampled)
Fastest is isChinese("ss扁担宽，板凳长，扁担想绑在板凳上。")
alsotang
The data structure looks asymmetric and this optimization is hard-coded. While the performance optimization of the proxy is pretty obvious, could it be a little more graceful?
alsotang issue alsotang/is_chinese_rs
alsotang
duplicate range
[0xf900, 0xfaff], // CJK Compatibility Ideographs is duplicate with [0xf900, 0xfaff], // https://en.wikipedia.org/wiki/CJK_Compatibility_Ideographs
alsotang issue comment alsotang/is_chinese_rs
alsotang
duplicate range
[0xf900, 0xfaff], // CJK Compatibility Ideographs is duplicate with [0xf900, 0xfaff], // https://en.wikipedia.org/wiki/CJK_Compatibility_Ideographs
alsotang
fixed by c51829061de2f36f9014ea0ff688afae2400bc72
alsotang push alsotang/is_chinese_rs
alsotang
remove duplicate char range
commit sha： c51829061de2f36f9014ea0ff688afae2400bc72
push time in 2 months agoalsotang push alsotang/is-chinese
alsotang
remove duplicate char range
commit sha： c34e415e33ce58e5ef1dc7a4cfd2b5c16ad87686
push time in 2 months ago
alsotang issue comment alsotang/is_chinese_rs
alsotang
duplicate range
[0xf900, 0xfaff], // CJK Compatibility Ideographs is duplicate with [0xf900, 0xfaff], // https://en.wikipedia.org/wiki/CJK_Compatibility_Ideographs
alsotang
Oh yes, I will fix it
Jul
30
2 months ago
Jul
27
2 months ago
alsotang push alsotang/socks5_rs
alsotang
TryFrom
commit sha： 4707b8607fdbfe069dd3acf0eaf03346e6505dc2
push time in 2 months ago
Jul
25
2 months ago
alsotang push alsotang/socks5_rs
alsotang
refactor
commit sha： 1e0d81722245dbaf7c31738158a8bb0096536ea7
push time in 2 months ago
Jul
24
3 months ago
reduce range of unicode
before:
after: