Compare commits

...

246 Commits

Author SHA1 Message Date
Yanyi Wu
294755fab1 build: refine CMakeLists.txt by removing unnecessary conditions and options
- Eliminated the default installation prefix condition to streamline the configuration.
- Simplified the test build logic by ensuring tests are enabled only for top-level projects.
- Cleaned up redundant code for better readability and maintainability.
2025-05-03 07:43:25 +08:00
Yanyi Wu
714a297823 build: update CMakeLists.txt to include additional directories for test configuration
- Added include directories for the current binary and test directories to improve test file accessibility.
- Ensured proper configuration for test paths in the build process.
2025-05-02 23:47:37 +08:00
Yanyi Wu
c14131e3e2 refactor: clean up load_test.cpp by removing unused dependencies and tests
- Removed unused Jieba test and associated includes from load_test.cpp.
- Simplified main function to focus on essential operations.
- Ensured consistent exit handling by returning EXIT_SUCCESS.
2025-05-02 23:41:53 +08:00
Yanyi Wu
9cd64a1694 build: enhance test configuration and path management
- Added configuration for test paths in CMake to simplify file references.
- Updated load_test.cpp and various unit tests to use defined path macros for dictionary and test data files.
- Introduced test_paths.h.in to manage directory paths consistently across tests.
2025-05-02 23:33:18 +08:00
Yanyi Wu
aa410a69bb build: simplify test configuration in CMakeLists.txt
- Removed conditional check for MSVC when adding test commands.
- Ensured that test commands are always added regardless of the compiler.
2025-05-02 21:39:18 +08:00
Yanyi Wu
b5dc8e7a35 build: update .gitignore and CMakeLists for test configuration
- Added entries to .gitignore for temporary test files.
- Included a message to display MSVC value during build.
- Added UTF-8 compile option for MSVC in unittest CMakeLists.
2025-05-02 21:28:28 +08:00
Yanyi Wu
8141d8f434
Merge pull request #200 from yanyiwu/dev
fix: remove outdated entry from jieba dictionary
2025-05-02 17:31:29 +08:00
yanyiwu
9d8af2116e build: update CI workflow to include latest OS versions 2025-05-02 11:53:33 +08:00
yanyiwu
2185315643 fix: remove outdated entry from jieba dictionary 2025-05-02 11:38:31 +08:00
yanyiwu
340de007f9 docs: update README.md 2025-04-13 18:59:44 +08:00
yanyiwu
940ea02eb4 deps: upgrade limonp from v1.0.0 to v1.0.1 2025-04-12 17:54:01 +08:00
yanyiwu
3732abc0e5 docs: update CHANGELOG for v5.5.0 2025-04-12 10:07:40 +08:00
yanyiwu
9cda7f33e8 build: upgrade googletest from 1.11.0 to 1.12.1 2025-04-12 10:02:10 +08:00
Yanyi Wu
338603b676
Merge pull request #196 from ahmadov/ahmadov/fix-ns-2
avoid implicit namespaces
2025-04-11 08:59:41 +08:00
Elmi Ahmadov
d93dda397c avoid implicit namespaces
This PR fixes the ambigious `partial_sort` in KeywordExtractor.hpp.
We also have a definition for it and the compiler is consufed which
implementation should be used. To fix it, we can use the `std` namespace
explicitly.

Also, use the `std` namespace for the other data structures and include
their headers.
2025-04-10 19:10:05 +02:00
Yanyi Wu
7730deee52
Merge pull request #195 from ahmadov/ahmadov/fix-ns
fix missing includes and make namespaces explicit
2025-04-10 23:01:18 +08:00
Elmi Ahmadov
588860b5b6 fix missing includes and make namespaces explicit 2025-04-10 16:11:20 +02:00
Yanyi Wu
0523949aa8
Update stale-issues.yml 2025-04-05 17:26:58 +08:00
Yanyi Wu
b11fd29697
Update README.md 2025-03-08 17:33:48 +08:00
yanyiwu
15b8086a2a Add CMake workflow for Windows ARM64 builds
This commit introduces a new GitHub Actions workflow for building and testing CMake projects on Windows ARM64. The workflow includes steps for checking out the repository, configuring CMake with multiple C++ standards, building the project, and running tests. This enhancement supports continuous integration for ARM64 architecture, improving the project's build versatility.
2025-01-18 20:58:17 +08:00
yanyiwu
1d74caf705 Update CMake minimum version requirement to 3.10 2025-01-18 20:47:06 +08:00
Yanyi Wu
0c7c5228d0
Update README.md 2025-01-17 23:47:09 +08:00
yanyiwu
016fc17575 Improve error logging for UTF-8 decoding failures across cppjieba components. Updated error messages in DictTrie, PosTagger, PreFilter, and SegmentBase to provide clearer context on the specific input causing the failure. This change enhances the debugging experience when handling UTF-8 encoded strings. 2024-12-08 17:26:28 +08:00
yanyiwu
39fc58f081 Remove macOS 12 from CI workflow in cmake.yml 2024-12-08 17:03:39 +08:00
yanyiwu
42a93a4b98 Refactor decoding functions to use UTF-8 compliant methods
Updated multiple files to replace instances of DecodeRunesInString with DecodeUTF8RunesInString, ensuring proper handling of UTF-8 encoded strings. This change enhances the robustness of string decoding across the cppjieba library, including updates in DictTrie, HMMModel, PosTagger, PreFilter, SegmentBase, and Unicode files. Additionally, corresponding unit tests have been modified to reflect these changes.
2024-12-08 16:46:24 +08:00
yanyiwu
5ee74d788e [stale-isssues] Monthly on the 3rd day of the month at midnight 2024-11-03 17:22:28 +08:00
yanyiwu
9b45e084a3 v5.4.0 2024-09-22 10:02:53 +08:00
yanyiwu
aa1def5ddb class Jiaba unittest add default argument input 2024-09-22 09:43:04 +08:00
yanyiwu
732812cdfb class Jieba: support default dictpath 2024-09-22 09:38:31 +08:00
yanyiwu
6e167a30dd cmake: avoid testing when FetchContent by other project 2024-09-22 00:25:23 +08:00
yanyiwu
5ef74f335a Revert "cmake: enable windows/msvc test"
This reverts commit 63392627552b018ea018848c82965c263b0030fa.
2024-09-21 23:58:59 +08:00
yanyiwu
6339262755 cmake: enable windows/msvc test 2024-09-21 21:49:56 +08:00
yanyiwu
cc58d4f858 DictTrie: removed unused var 2024-09-21 21:29:55 +08:00
yanyiwu
dbebc7cacb cmake: enable windows/msvc test 2024-09-21 21:10:53 +08:00
yanyiwu
e5b98af199 v5.3.2 2024-09-21 20:45:46 +08:00
yanyiwu
e521f26456 removed test/demo.cpp and linked https://github.com/yanyiwu/cppjieba-demo 2024-09-21 17:26:19 +08:00
Yanyi Wu
30aaf7b9ad
Update Demo Link in README.md 2024-09-21 17:21:54 +08:00
yanyiwu
84bca4bc50 [github/actions] stale 1 year ago issues 2024-09-14 21:49:46 +08:00
yanyiwu
3c8663472b [github/actions] stale 3 years ago issues 2024-09-14 21:37:25 +08:00
yanyiwu
12341a2f21 [stale issues] Run weekly on Sunday at midnight 2024-09-11 21:41:15 +08:00
yanyiwu
165bee901c [github/actions] stale issues 2024-09-07 20:58:52 +08:00
yanyiwu
e691b631b2 limonp v0.9.0 -> v1.0.0 2024-09-07 17:21:59 +08:00
yanyiwu
31dfe0f9d0 v5.3.1 2024-08-17 17:21:44 +08:00
yanyiwu
a110ab10cc [cmake] fetch googletest 2024-08-16 10:13:07 +08:00
yanyiwu
fe88bd29ac [submodules] rm test/googletest 2024-08-16 10:08:36 +08:00
yanyiwu
00c8f8fa84 v5.3.0 2024-08-10 22:00:50 +08:00
yanyiwu
90174da597 [c++17,c++20] compatibility 2024-08-10 21:50:01 +08:00
yanyiwu
a7adc22a6e limonp version 0.6.7 -> 0.9.0 2024-08-10 21:47:27 +08:00
yanyiwu
30ab2c3860 v5.2.0 2024-07-28 23:25:58 +08:00
yanyiwu
3748b56928 [README] platform updated 2024-07-28 23:18:53 +08:00
yanyiwu
79a235223d [CI] windows-[2019,2022] 2024-07-28 23:16:11 +08:00
yanyiwu
c39fd30f93 [googletest] v1.6.0->v1.10.0 2024-07-28 22:41:20 +08:00
yanyiwu
f4c87c2ff4 [CI] ubuntu version from 20 to 22, macos version from 12 to 14 2024-07-28 22:32:46 +08:00
yanyiwu
f8d063101c [CMake] mini_required 2.6->3.5 and fix CXX_VERSION variable passed from cmd 2024-07-27 19:24:57 +08:00
yanyiwu
732fec41e6 [CI] matrix and multi cpp version 2024-07-27 18:58:36 +08:00
yanyiwu
bc162dbd84 v5.1.3 2024-07-22 22:53:53 +08:00
yanyiwu
8aad517375 git submodule add googletest-1.6.0 2024-07-22 22:36:45 +08:00
yanyiwu
4ec3204280 [Changelog] v5.1.2 2024-07-16 07:27:24 +08:00
yanyiwu
c83c7111ab README fix typo 2024-07-15 22:56:01 +08:00
yanyiwu
e334fc2ce0 [submodule:deps/limonp] upgrade to v0.6.7 2024-07-14 22:58:39 +08:00
yanyiwu
bc90a8276e rm useless code 2024-07-14 09:47:34 +08:00
yanyiwu
4b2f257a6a README_EN.md is useless for AI-age 2024-07-14 00:09:49 +08:00
unknown
7cf4502e01 commit from vscode 2024-07-13 20:03:58 +08:00
Yanyi Wu
a4e2b67017
Update README.md for windows env 2024-07-13 17:15:12 +08:00
Yanyi Wu
c22b6843d3
Update ChangeLog.md 2024-06-07 17:19:23 +08:00
Yanyi Wu
f4145fd08e
Merge pull request #186 from appotry/master
修正编译报错
2024-05-01 17:05:12 +08:00
夜法之书(appotry)
f91776997b
修正编译报错 2024-05-01 16:06:53 +08:00
wuyanyi
391121d5db release v5.1.0 2022-10-16 13:25:50 +08:00
Yanyi Wu
d869831996
Merge pull request #172 from yanyiwu/wyy
feature: add RemoveWord api from gojieba/pull/99
2022-10-16 13:24:40 +08:00
wuyanyi
03cc7c39ff feature: add RemoveWord api from https://github.com/yanyiwu/gojieba/pull/99 2022-10-16 13:17:19 +08:00
wuyanyi
302a367338 release v5.0.5 2022-10-16 12:43:01 +08:00
Yanyi Wu
1bf8e11833
Merge pull request #171 from yanyiwu/wyy
[submodule] update limonp to v0.6.6
2022-10-16 12:26:00 +08:00
wuyanyi
fc6e3f4294 [submodule] update limonp to v0.6.6 2022-10-16 12:10:35 +08:00
wuyanyi
db9b4b6813 [release v5.0.4] 2022-10-16 11:37:16 +08:00
Yanyi Wu
99b496d871
Merge pull request #168 from playgithub/limonp-as-submodule
limonp as submodule
2022-07-31 22:21:03 +08:00
abc
3269637644 remove .travis.yml 2022-07-31 21:25:29 +08:00
abc
fe9901858c remove appveyor.yml 2022-07-31 21:17:06 +08:00
abc
6a1d49d99b Update github workflows to checkout code with submodules 2022-07-22 10:41:50 +08:00
abc
8c93e0978d Update CMakeLists.txt to use limonp as a submodule 2022-07-22 10:36:35 +08:00
abc
01aba1d85d add submodule limonp 2022-07-22 10:34:52 +08:00
Yanyi Wu
194c144d8b
Merge pull request #166 from playgithub/add-license
Add MIT license
2022-07-10 21:51:23 +08:00
abc
1da8e7cde7 Fix end of line sequence, use LF instead of CRLF 2022-07-10 16:38:27 +08:00
abc
23ce5c7050 Add MIT license 2022-07-10 16:28:09 +08:00
Yanyi Wu
ef2b8f8b1b
Update README.md 2022-01-31 16:44:50 +08:00
Yanyi Wu
e81930b7c2
Update README.md 2022-01-09 21:07:50 +08:00
Yanyi Wu
466419dda0
Create cmake.yml 2022-01-09 18:25:10 +08:00
Yanyi Wu
acb2ecc125
Merge pull request #151 from wangfenjin/patch-2
add simple: sqlite3 fts5 tokenizer
2021-02-21 11:35:57 +08:00
Wang Fenjin
677b471d9f
add simple: sqlite3 fts5 tokenizer 2021-02-20 18:48:23 +08:00
Yanyi Wu
7a61f14b6b
Update README.md 2020-11-21 22:22:36 +08:00
Yanyi Wu
b799936d2f
Update README.md 2020-06-23 09:34:28 +08:00
Yanyi Wu
f66c9d1184
Merge pull request #145 from yanyiwu/yanyiwu-patch-1-1
add sponsorship
2020-06-21 01:15:59 +08:00
Yanyi Wu
1d4ffd4b3d
add sponsorship 2020-06-21 00:56:57 +08:00
yanyiwu
32fd1ef010 [release] v5.0.3 2020-03-11 09:30:52 +08:00
yanyiwu
7c046e393f Upgrade [limonp](https://github.com/yanyiwu/limonp) -> v0.6.3 2020-03-11 09:23:22 +08:00
yanyiwu
b6a1f5f21c [release] v5.0.2 2020-01-13 21:33:53 +08:00
yanyiwu
dfb9c1f010 Upgrade [limonp](https://github.com/yanyiwu/limonp) -> v0.6.1 2020-01-13 21:29:53 +08:00
Yanyi Wu
79ffd00979
Update README.md 2019-10-24 23:05:33 +08:00
yanyiwu
caf4c43ad6 release tag v5.0.1 2019-09-21 12:01:15 +08:00
Yanyi Wu
9aa7537096
Update README.md 2019-09-21 11:57:15 +08:00
Yanyi Wu
28b37b9cba
Merge pull request #134 from shove70/patch-1
Explanation for adding D language bindings.
2019-09-21 09:51:00 +08:00
shove70
4f04293261 Explanation for adding D language bindings. 2019-09-21 09:28:06 +08:00
Yanyi Wu
be6e91e6c3
Update README.md 2019-09-15 18:14:02 +08:00
Yanyi Wu
8a258dfaf4
Merge pull request #127 from byronhe/patch-2
remove duplicate #include
2019-09-15 16:54:42 +08:00
Yanyi Wu
f39192e983
Merge pull request #133 from byronhe/patch-5
fix typo
2019-09-04 22:54:46 +08:00
byronhe
55a94b417c
fix typo 2019-09-04 20:50:11 +08:00
Yanyi Wu
866d0e83b0
Merge pull request #129 from byronhe/patch-4
fix compile warning
2019-04-29 12:27:41 +08:00
byronhe
6444f4b226
fix compile warning 2019-04-29 12:18:03 +08:00
Yanyi Wu
7fc865760b
Merge pull request #128 from byronhe/patch-3
会导致别的含有 print 这个符号的代码编译不过
2019-03-16 13:46:52 +08:00
byronhe
f55b591968
会导致别的含有 print 这个符号的代码编译不过
会导致别的含有 print 这个符号的代码编译不过
2019-03-15 22:02:04 +08:00
byronhe
798b7b81c9
remove duplicate #include
remove duplicate #include
2019-03-15 15:48:09 +08:00
Yanyi Wu
8fca7300a4
Merge pull request #126 from maliubiao/master
修正c++版本判断兼容性问题
2019-03-02 23:42:53 +08:00
maliubiao
07382b9cb1
修正c++版本判断兼容性问题
c++11以上,这个分支会跳到错误的地方,错误的使用using std::tr1::unordered_map ,导致undefined symbol.
2019-03-02 16:02:02 +08:00
Yanyi Wu
31eed03518
Update README.md 2018-10-05 11:33:48 +08:00
Yanyi Wu
3dfdc426f0
Update README.md 2018-10-05 11:27:35 +08:00
Yanyi Wu
7b2fdc41a2
Merge pull request #113 from bung87/exposes_InsertUserWord_and_Find
Exposes insert user word and find
2018-06-09 19:51:12 +08:00
zhoupeng
985ccd646c Merge branch 'master' of https://github.com/yanyiwu/cppjieba into HEAD 2018-06-09 16:23:49 +08:00
zhoupeng
111fb007cf exposes InsertUserWord Find 2018-06-09 16:21:13 +08:00
Yanyi Wu
e6fdd1c98b
Merge pull request #112 from bung87/master
接口一致与完整
2018-06-08 23:13:44 +08:00
zhoupeng
1e1e585194 LoadUserDict by set,vector 2018-06-08 14:23:01 +08:00
zhoupeng
1066bc085e fix input type ,expose to Jieba 2018-06-08 01:32:47 +08:00
zhoupeng
d56e5c0659 InsertUserWord with freq arg,expose InserUserDictNode with vector<string> arg 2018-06-08 00:44:33 +08:00
Yanyi Wu
36be7fb900
Merge pull request #111 from bung87/master
增加cppjieba-py 扩展说明
2018-06-07 23:19:06 +08:00
Yanyi Wu
bd368bc04d
Merge pull request #105 from Silencezjl/patch-2
Update demo.cpp
2018-06-07 23:18:35 +08:00
zhoupeng
cb4011ac56 增加cppjieba-py 扩展说明 2018-06-07 16:52:06 +08:00
张家麟
1089dcdcd3
Update demo.cpp
多了个分号~ 虽然影响不大~
2018-01-29 10:12:38 +00:00
Yanyi Wu
6aff1f637c Merge pull request #96 from wangzhe258369/master
减少Visual Studio编译器警告
2017-06-28 00:02:16 +08:00
Wangzhe
e7602afaac 减少Visual Studio编译器警告 2017-06-27 23:00:31 +08:00
yanyiwu
dabe502bb4 fix travis compiler 2017-04-03 23:14:59 +08:00
Yanyi Wu
d42602c12d Merge pull request #88 from stphnlyd/readme
mention the Perl 5 binding for CppJieba
2017-04-03 22:56:58 +08:00
Stephan Loyd
3d04caa1b1 mention Perl 5 binding for CppJieba 2017-04-03 22:37:04 +08:00
Yanyi Wu
472a584487 Merge pull request #87 from jonnywang/patch-2
增加php扩展版本链接
2017-03-31 10:38:40 +08:00
星期八
27dbfb8146 增加php扩展版本链接 2017-03-31 10:24:39 +08:00
Yanyi Wu
e5d9eb8816 Merge pull request #79 from royguo/master
Add Unicode offset/length support for `Word`
2016-10-18 23:02:01 +08:00
Roy Guo
f74d716570 Add Unicode offset/length support for Word 2016-10-16 13:05:56 +08:00
Roy Guo
a2f75a00d3 Add Unicode offset/length support for Word 2016-10-16 12:52:50 +08:00
yanyiwu
45809955f5 v5.0.0 2016-09-11 21:44:51 +08:00
yanyiwu
74c70c70cd create keyword_extract in Jieba 2016-09-11 21:42:53 +08:00
yanyiwu
4a755dff6a may be more friendly for compiler 2016-08-11 00:00:20 +08:00
yanyiwu
53bc279dea fix compiler warning 2016-07-23 20:49:27 +08:00
yanyiwu
91b7f9af63 v4.8.1 2016-07-23 00:11:02 +08:00
yanyiwu
0984c9ed3f update user dict loading method about word weight, and add unit tests 2016-07-22 23:53:49 +08:00
Yanyi Wu
e45ac012cb Merge pull request #74 from npes87184/master
fix second element parse error in dict
2016-07-22 13:40:55 +08:00
npes87184
0c3cf04b43 fix second element parse error in dict 2016-07-22 10:19:28 +08:00
Yanyi Wu
e3e5f93ca3 Merge pull request #73 from bigelephant29/user-dict-tag-bug-fix
fix user dict tag bug : wrong buf index assigned
2016-07-21 12:26:16 +08:00
bigelephant29
986106a553 change stoi to atoi 2016-07-21 10:54:08 +08:00
bigelephant29
2e1b6e0443 user dict support user weight and user tag 2016-07-21 10:38:46 +08:00
bigelephant29
b82acaf71e fix user dict tag bug : wrong buf index assigned 2016-07-21 10:06:24 +08:00
Yanyi Wu
8b75bf14a3 Merge pull request #72 from t-k-/master
增加 LookupTag 函数来对单个的 token 进行 tag 查询
2016-07-07 11:15:59 +08:00
t-k-
e40270ca86 Avoid using `initializer lists' from C++0x. 2016-07-06 13:48:18 -06:00
t-k-
5775a40bee Add LookupTag function for single token tag lookup. 2016-07-06 02:44:56 -06:00
Yanyi Wu
667acdeb7b Merge pull request #71 from jaiminpan/master
add tag capbility for each segments
2016-07-03 20:10:49 +08:00
Jaimin Pan
ce8cafe54a add tag capbility for each segments 2016-06-27 18:10:42 +08:00
yanyiwu
ec848581b2 fix issue #70 2016-06-10 21:49:31 +08:00
Yanyi Wu
0bf9341dd6 Merge pull request #69 from vsooda/master
fix unittest cmake macro bug
2016-06-08 11:01:47 +08:00
sooda
7d503e4b13 fix unittest cmake macro bug 2016-06-08 10:38:20 +08:00
yanyiwu
c0afac2598 update changelog 2016-05-09 22:52:42 +08:00
yanyiwu
c425bcc49f add Jieba::ResetSeparators api and unittest 2016-05-09 22:49:51 +08:00
yanyiwu
6e3ecec599 improve readability 2016-05-09 22:09:57 +08:00
yanyiwu
e4e1b4e953 update readme 2016-05-09 21:23:05 +08:00
Yanyi Wu
02df433f73 Merge pull request #65 from questionfish/master
增加了TextRank关键词提取
2016-05-04 20:02:07 +08:00
Yanyi Wu
00b2eb13c6 Merge pull request #2 from yanyiwu/patch-1
Patch 1
2016-05-04 19:33:37 +08:00
yanyiwu
b355e9f487 update unittest to pass 'make test' 2016-05-04 19:33:05 +08:00
yanyiwu
0a23d6b268 merge questionfish/master 2016-05-04 19:27:05 +08:00
mayunyun
d5a52a8e7b 1. remove stopword from span windows
2. update unittest
2016-05-04 17:52:30 +08:00
yanyiwu
5c739484ae merge the latest codes in master branch, and update unittest cases to pass ci 2016-05-03 23:20:03 +08:00
questionfish
04c176de08 Merge pull request #1 from yanyiwu/patch-1
Update TextRankExtractor.hpp: use yanyiwu's correction
2016-05-03 21:46:01 +08:00
yanyiwu
f253db0133 use map/set instead of unordered_map/unordered_set to make result stable 2016-05-03 21:24:40 +08:00
yanyiwu
39316114c5 correct unittest case 2016-05-03 20:49:47 +08:00
yanyiwu
a1ea1d0757 add textrank unittest into cmake 2016-05-03 20:01:44 +08:00
Yanyi Wu
6d105a864d Update TextRankExtractor.hpp
remove unused function which using c++11 keyword `auto`
2016-05-03 19:53:40 +08:00
mayunyun
0f66a923b3 1.增加单元测试
2.增加了构造函数的重载,增加了提取函数的重载
2016-05-03 18:06:14 +08:00
mayunyun
f2de41c15e code layout change: tab -> space 2016-05-03 09:03:16 +08:00
yanyiwu
a778d47046 v4.8.0 2016-05-02 17:15:38 +08:00
yanyiwu
5ac9e48eb0 rewrite QuerySegment, make Jieba::CutForSearch behaves the same as [jieba] cut_for_search api
remove Jieba::SetQuerySegmentThreshold
2016-05-02 16:18:36 +08:00
yanyiwu
3f0faec14b windows ci test 2016-04-27 20:22:05 +08:00
Yanyi Wu
4d8d793da5 Merge pull request #63 from qinwf/windows-appveyor
add Windows CI with MSVC
2016-04-27 19:13:49 +08:00
qinwf
c84594f620 add Windows CI with MSVC 2016-04-27 17:45:48 +08:00
yanyiwu
e6074eecb9 add cppjieba-server link 2016-04-27 16:24:13 +08:00
mayunyun
1aa0a32d90 code format check 2016-04-25 20:28:47 +08:00
mayunyun
669e971e3e new file: include/cppjieba/TextRankExtractor.hpp
Add TextRank Keyword Extractor to JiebaCpp
新增TextRank关键词提取
2016-04-25 20:20:50 +08:00
yanyiwu
d9e8cdac36 v4.7.0 2016-04-21 14:28:02 +08:00
yanyiwu
9ebc906d3f update README 2016-04-19 16:04:44 +08:00
yanyiwu
3befc42697 update KeywordExtractor::Word's printing format to json format 2016-04-19 16:00:53 +08:00
yanyiwu
a9301facde upgrade limonp -> v0.6.1 2016-04-19 15:24:56 +08:00
yanyiwu
29e085904d add log and unittest 2016-04-18 14:55:42 +08:00
yanyiwu
63e9c94fb7 add unicode decoding unittest 2016-04-18 14:37:17 +08:00
yanyiwu
6fa843b527 override Cut functions, add location information into Word results; 2016-04-17 23:39:57 +08:00
yanyiwu
b6703aba90 use offset instead of str in RuneStr 2016-04-17 22:50:32 +08:00
yanyiwu
e7a45d2dde remove LevelSegment 2016-04-17 22:23:00 +08:00
yanyiwu
42a73eeb64 make compiler happy 2016-04-17 22:11:58 +08:00
yanyiwu
dcced8561e remove namespace unicode 2016-04-17 21:59:10 +08:00
yanyiwu
6ff6fe1430 WordRange construct 2016-04-17 21:57:36 +08:00
yanyiwu
339e3ca772 big change: add RuneStr for the position of word in string 2016-04-17 17:30:05 +08:00
yanyiwu
abcc0af034 update readme 2016-03-30 00:41:44 +08:00
Yanyi Wu
1fb5a7c66f Update README.md 2016-03-29 23:50:59 +08:00
Yanyi Wu
82feba693c Merge pull request #59 from bitdeli-chef/master
Add a Bitdeli Badge to README
2016-03-29 00:52:05 -05:00
Bitdeli Chef
627a514b7f Add a Bitdeli badge to README 2016-03-29 06:04:47 +00:00
yanyiwu
81cd435f2a prettify demo output 2016-03-28 01:22:24 +08:00
yanyiwu
500af453e1 add new case: sqljieba 2016-03-27 23:46:14 +08:00
yanyiwu
4b97c57bb2 v4.6.0 2016-03-26 23:34:40 +08:00
yanyiwu
c19736995c Add KeywordExtractor::Word and add more overrided KeywordExtractor::Extract 2016-03-26 22:12:40 +08:00
yanyiwu
e6a2b47b87 hange the return value of KeywordExtractor::Extract from bool to void 2016-03-26 01:16:44 +08:00
yanyiwu
5102b8a5c3 Change Jieba::Locate to be static function. 2016-03-26 01:14:48 +08:00
yanyiwu
7db3f87b5f remove info log for dict loading 2016-03-22 10:45:20 +08:00
yanyiwu
5a8a0fae7a v4.5.3 2016-03-18 16:17:57 +08:00
yanyiwu
3ef005275a Upgrade limonp to v0.6.0 2016-03-18 16:14:48 +08:00
yanyiwu
81c35dde01 v4.5.2 2016-03-18 14:32:52 +08:00
yanyiwu
92fdf009cb Upgrade limonp to v0.5.6 to fix hidden trouble. 2016-03-18 14:05:18 +08:00
yanyiwu
643148edf5 platform 2016-02-26 22:33:51 +08:00
yanyiwu
f446ecf2ed v4.5.1 2016-02-19 16:29:59 +08:00
yanyiwu
3e28b4bcb1 adjust code for limonp v0.5.5 to solve macro name conflicts 2016-02-19 16:15:23 +08:00
yanyiwu
fc04cf750a upgrade limonp to v0.5.5 2016-02-19 16:14:33 +08:00
yanyiwu
9d7da4864a v4.5.0 2016-02-18 16:21:15 +08:00
yanyiwu
0a7b6e62f3 add Unicode32 cases for cut testing 2016-02-18 15:18:35 +08:00
yanyiwu
14e09290c2 change Rune type from uint16_t to uint32_t to support more chinese word 2016-02-18 14:54:03 +08:00
yanyiwu
8d66b1f1fa upgrade limonp to v0.5.4 2016-02-18 14:48:26 +08:00
yanyiwu
239d025cd8 delete HashMap, use unordered_map instead 2016-02-16 20:24:28 +08:00
yanyiwu
e6454fef77 use HashMap in Trie, and remove the base array of trie root node, see details in Changelog 2016-02-12 01:37:39 +08:00
yanyiwu
2d3c51dba7 upgrade limonp and use limonp::HashMap in Trie 2016-02-04 23:43:26 +08:00
yanyiwu
6f303ee843 v4.4.1 2016-01-29 10:14:40 +08:00
yanyiwu
8496f41e5d update changelog.md 2016-01-29 00:49:29 +08:00
yanyiwu
721b34f1bd fix bug, see details in ChangeLog.md 2016-01-29 00:30:38 +08:00
Yanyi Wu
8ca338d75a Update README.md 2016-01-22 21:38:53 +08:00
yanyiwu
446c21851d v4.4.0 2016-01-21 22:19:10 +08:00
Yanyi Wu
550ac2ab61 Merge pull request #52 from yanyiwu/remove_server
remove server, see details in ChangeLog.md
2016-01-21 19:12:54 +08:00
yanyiwu
34668aa379 remove server, see details in ChangeLog.md 2016-01-21 01:07:31 +08:00
yanyiwu
c1a6726bcc update readme.md 2016-01-20 20:47:21 +08:00
yanyiwu
963bf516a6 v4.3.3 2016-01-20 12:21:08 +08:00
yanyiwu
4493c604b9 Yet Another Incompatibility Problem Repair: Upgrade [limonp] to version v0.5.3, fix incompatibility problem in Windows 2016-01-20 12:06:56 +08:00
yanyiwu
c34c8f3082 v4.3.2 2016-01-16 01:44:28 +08:00
yanyiwu
0482ec2b6c [limonp] to version v0.5.2, fix incompatibility problem in Windows 2016-01-16 01:34:18 +08:00
yanyiwu
eb12813194 v4.3.1 2016-01-13 00:43:11 +08:00
yanyiwu
193e717d22 override constructor in KeywordExtractor 2016-01-13 00:40:46 +08:00
yanyiwu
a6c6e8df8c v4.3.0 2016-01-11 15:02:09 +08:00
yanyiwu
b41cb0e2ee fix compile error 2016-01-11 14:50:14 +08:00
yanyiwu
d92d3f194d upgrade husky to version v0.2.2 2016-01-11 14:33:43 +08:00
yanyiwu
3ab9a34909 upgrade limonp to version v0.5.1 2016-01-11 14:30:38 +08:00
yanyiwu
3c5ad24260 source code layout change:
1. src/ -> include/cppjieba/
2. src/limonp/ -> deps/limonp/
3. server/husky -> deps/husky/
4. test/unittest/gtest -> deps/gtest
2016-01-11 14:25:02 +08:00
yanyiwu
a07a22e9c4 update README 2016-01-10 19:58:47 +08:00
yanyiwu
a740fca866 add english readme 2015-12-24 21:35:06 +08:00
yanyiwu
29306c977f add badge 2015-12-24 21:09:10 +08:00
yanyiwu
fb5d989dc6 v4.2.1 2015-12-12 21:26:45 +08:00
yanyiwu
bcb112a4b1 upgrade basic functions 2015-12-12 21:25:57 +08:00
yanyiwu
8bf70127c2 upgrade limonp to version v0.4.1 2015-12-12 21:02:40 +08:00
yanyiwu
484ce39d36 update husky to version v0.2.0 2015-12-12 19:43:49 +08:00
yanyiwu
194550823f update limonp to version v0.4.0 2015-12-12 19:42:30 +08:00
126 changed files with 2482 additions and 35715 deletions

40
.github/workflows/cmake-arm64.yml vendored Normal file
View File

@ -0,0 +1,40 @@
name: CMake Windows ARM64
on:
push:
pull_request:
workflow_dispatch:
env:
BUILD_TYPE: Release
jobs:
build-windows-arm64:
runs-on: windows-2022
strategy:
matrix:
cpp_version: [11, 14, 17, 20]
steps:
- name: Check out repository code
uses: actions/checkout@v2
with:
submodules: recursive
- name: Configure CMake
# Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make.
# See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type
# run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}
run: cmake -B ${{github.workspace}}/build -DBUILD_TESTING=ON -DCMAKE_CXX_STANDARD=${{matrix.cpp_version}} -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}
- name: Build
# Build your program with the given configuration
# run: cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}
run: cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}
- name: Test
working-directory: ${{github.workspace}}/build
# Execute tests defined by the CMake configuration.
# See https://cmake.org/cmake/help/latest/manual/ctest.1.html for more detail
run: ctest -C ${{env.BUILD_TYPE}} --verbose

53
.github/workflows/cmake.yml vendored Normal file
View File

@ -0,0 +1,53 @@
name: CMake
on:
push:
pull_request:
env:
# Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
BUILD_TYPE: Release
jobs:
build:
# The CMake configure and build commands are platform agnostic and should work equally well on Windows or Mac.
# You can convert this to a matrix build if you need cross-platform coverage.
# See: https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [
ubuntu-22.04,
ubuntu-latest,
macos-13,
macos-14,
macos-latest,
windows-2019,
windows-2022,
windows-latest,
]
cpp_version: [11, 14, 17, 20]
steps:
- name: Check out repository code
uses: actions/checkout@v2
with:
submodules: recursive
- name: Configure CMake
# Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make.
# See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type
# run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}
run: cmake -B ${{github.workspace}}/build -DBUILD_TESTING=ON -DCMAKE_CXX_STANDARD=${{matrix.cpp_version}} -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}
- name: Build
# Build your program with the given configuration
# run: cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}
run: cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}
- name: Test
working-directory: ${{github.workspace}}/build
# Execute tests defined by the CMake configuration.
# See https://cmake.org/cmake/help/latest/manual/ctest.1.html for more detail
run: ctest -C ${{env.BUILD_TYPE}} --verbose

25
.github/workflows/stale-issues.yml vendored Normal file
View File

@ -0,0 +1,25 @@
name: Close Stale Issues
on:
schedule:
- cron: '0 0 3 */3 *' # Every three months on the 3rd day at midnight
jobs:
stale:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v5
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: 'This issue has not been updated for over 1 year and will be marked as stale. If the issue still exists, please comment or update the issue, otherwise it will be closed after 7 days.'
close-issue-message: 'This issue has been automatically closed due to inactivity. If the issue still exists, please reopen it.'
days-before-issue-stale: 365
days-before-issue-close: 7
stale-issue-label: 'Stale'
exempt-issue-labels: 'pinned,security'
operations-per-run: 100

2
.gitignore vendored
View File

@ -15,3 +15,5 @@ tmp
t.*
*.pid
build
Testing/Temporary/CTestCostData.txt
Testing/Temporary/LastTest.log

3
.gitmodules vendored Normal file
View File

@ -0,0 +1,3 @@
[submodule "deps/limonp"]
path = deps/limonp
url = https://github.com/yanyiwu/limonp.git

View File

@ -1,22 +0,0 @@
language: cpp
before_install:
- if [ $TRAVIS_OS_NAME == linux ]; then sudo apt-get install cmake; fi
- if [ $TRAVIS_OS_NAME == osx ]; then brew install cmake; fi
script:
- mkdir build
- cd build
- cmake ..
- make
- make test
os:
- linux
- osx
compiler:
- clang
- gcc
notifications:
recipients:
- i@yanyiwu.com
email:
on_success: change
on_failure: always

View File

@ -1,4 +1,175 @@
# CppJieba ChangeLog
# CHANGELOG
## v5.5.0
+ feat: add Windows ARM64 build support
+ build: upgrade googletest from 1.11.0 to 1.12.1
+ build: update CMake minimum version requirement to 3.10
+ fix: make namespaces explicit and fix missing includes
+ ci: update stale-issues workflow configuration
## v5.4.0
+ unittest: class Jiaba add default argument input
+ class Jieba: support default dictpath
+ cmake: avoid testing when FetchContent by other project
+ class DictTrie: removed unused var
## v5.3.2
+ removed test/demo.cpp and linked https://github.com/yanyiwu/cppjieba-demo
+ Update Demo Link in README.md
+ [github/actions] stale 1 year ago issues
+ limonp v0.9.0 -> v1.0.0
## v5.3.1
+ [cmake] fetch googletest
+ [submodules] rm test/googletest
## v5.3.0
+ [c++17,c++20] compatibility
+ limonp version 0.6.7 -> 0.9.0
## v5.2.0
+ [CI] windows-[2019,2022]
+ [googletest] v1.6.0->v1.10.0
+ [CI] ubuntu version from 20 to 22, macos version from 12 to 14
+ [CMake] mini_required 2.6->3.5 and fix CXX_VERSION variable passed from cmd
+ [CI] matrix and multi cpp version [11, 14]
## v5.1.3
+ [googletest] git submodule add googletest-1.6.0
## v5.1.2
+ [submodule:deps/limonp] upgrade to v0.6.7
## v5.1.1
+ Merged [pr-186](https://github.com/yanyiwu/cppjieba/pull/186)
## v5.1.0
+ Merged [feature: add RemoveWord api from gojieba/pull/99 #172](https://github.com/yanyiwu/cppjieba/pull/172)
## v5.0.5
+ Merged [pr-171 submodule update limonp to v0.6.6 #171](https://github.com/yanyiwu/cppjieba/pull/171)
## v5.0.4
+ Merged [pr-168 limonp as submodule #168](https://github.com/yanyiwu/cppjieba/pull/168)
## v5.0.3
+ Upgrade [limonp](https://github.com/yanyiwu/limonp) -> v0.6.3
## v5.0.2
+ Upgrade [limonp](https://github.com/yanyiwu/limonp) -> v0.6.1
## v5.0.1
+ Make Compiler Happier.
+ Add PHP, DLang Links.
## v5.0.0
+ Notice(**api changed**) : Jieba class 3 arguments -> 5 arguments, and use KeywordExtractor in Jieba
## v4.8.1
+ add TextRankExtractor by [@questionfish] in [pull request 65](https://github.com/yanyiwu/cppjieba/pull/65)
+ add Jieba::ResetSeparators api for some special situation, for example in [issue67](https://github.com/yanyiwu/cppjieba/issues/67)
+ fix [issue70](https://github.com/yanyiwu/cppjieba/issues/70)
+ support (word, freq, tag) format in user_dict, see details in [pr74](https://github.com/yanyiwu/cppjieba/pull/74)
## v4.8.0
+ rewrite QuerySegment, make `Jieba::CutForSearch` behaves the same as [jieba] `cut_for_search` api
+ remove Jieba::SetQuerySegmentThreshold
## v4.7.0
api changes:
+ override Cut functions, add location information into Word results;
+ remove LevelSegment;
+ remove Jieba::Locate;
upgrade:
+ limonp -> v0.6.1
## v4.6.0
+ Change Jieba::Locate(deprecated) to be static function.
+ Change the return value of KeywordExtractor::Extract from bool to void.
+ Add KeywordExtractor::Word and add more overrided KeywordExtractor::Extract
## v4.5.3
+ Upgrade limonp to v0.6.0
## v4.5.2
+ Upgrade limonp to v0.5.6 to fix hidden trouble.
## v4.5.1
+ Upgrade limonp to v0.5.5 to solve macro name conficts in some special case.
## v4.5.0
+ 在 Trie 中去除之前糟糕的针对 uint16 优化的用数组代替 map 的设计,
该设计的主要问题是前提 unicode 每个字符必须是 uint16 ,则无法更全面得支持 unicode 多国字符。
+ Rune 类型从 16bit 更改为 32bit ,支持更多 Unicode 字符,包括一些罕见汉字。
## v4.4.1
+ 使用 valgrind 检查内存泄露的问题定位出一个HMM模型初始化的问题导致内存泄露的bug不过此内存泄露不是致命问题
因为只会在词典载入的时候发生,而词典载入通常情况下只会被运行一次,故不会导致严重问题。
+ 感谢 [qinwf] 帮我发现这个bug非常感谢。
## v4.4.0
+ 加代码容易删代码难,思索良久,还是决定把 Server 功能的源码剥离出这个项目。
+ 让 [cppjieba] 回到当年情窦未开时清纯的感觉删除那些无关紧要的server代码让整个项目轻装上阵专注分词的核心代码。
+ By the way, 之前的 server 相关的代码,如果你真的需要它,就去新的项目仓库 [cppjieba-server](https://github.com/yanyiwu/cppjieba-server) 找它们。
## v4.3.3
+ Yet Another Incompatibility Problem Repair: Upgrade [limonp] to version v0.5.3, fix incompatibility problem in Windows
## v4.3.2
+ Upgrade [limonp] to version v0.5.2, fix incompatibility problem in Windows
## v4.3.1
+ 重载 KeywordExtractor 的构造函数,可以传入 Jieba 进行字典和模型的构造。
## v4.3.0
源码目录布局调整:
1. src/ -> include/cppjieba/
2. src/limonp/ -> deps/limonp/
3. server/husky -> deps/husky/
4. test/unittest/gtest -> deps/gtest
依赖库升级:
1. [limonp] to version v0.5.1
2. [husky] to version v0.2.0
## v4.2.1
1. Upgrade [limonp] to version v0.4.1, [husky] to version v0.2.0
## v4.2.0
@ -66,14 +237,14 @@
## v2.4.3
1. 更新 [Husky] 服务代码,新 [Husky] 为基于线程池的服务器简易框架。并且修复当 HTTP POST 请求时 body 过长数据可能丢失的问题。
1. 更新 [husky] 服务代码,新 [husky] 为基于线程池的服务器简易框架。并且修复当 HTTP POST 请求时 body 过长数据可能丢失的问题。
2. 修改 PosTagger 的参数结构,删除暂时无用的参数。并添加使用自定义字典的参数,也就是支持 **自定义词性**
3. 更好的支持 `mac osx` (原谅作者如此屌丝,这么晚才买 `mac` )。
4. 支持 `Docker` ,具体请见 `Dockerfile`
## v2.4.2
1. 适当使用 `vector` 的基础上,使用`Limonp/LocalVector.hpp`作为`Unicode`的类型等优化,约提高性能 `30%`
1. 适当使用 `vector` 的基础上,使用`limonp/LocalVector.hpp`作为`Unicode`的类型等优化,约提高性能 `30%`
2. 使 `cjserver` 支持用户自定义词典,通过在 `conf/server.conf` 里面配置 `user_dict_path` 来实现。
3. 修复 `MPSegment` 切词时,当句子中含有特殊字符时,切词结果不完整的问题。
4. 修改 `FullSegment` 减少内存使用。
@ -131,9 +302,12 @@
1. 完成__最大概率分词算法__和__HMM分词算法__并且将他们结合起来成效果最好的`MixSegment`
2. 进行大量的代码重构将主要的功能性代码都写成了hpp文件。
3. 使用`cmake`工具来管理项目。
4. 使用 [Limonp]作为工具函数库,比如日志,字符串操作等常用函数。
5. 使用 [Husky] 搭简易分词服务的服务器框架。
4. 使用 [limonp]作为工具函数库,比如日志,字符串操作等常用函数。
5. 使用 [husky] 搭简易分词服务的服务器框架。
[Limonp]:http://github.com/yanyiwu/limonp.git
[Husky]:http://github.com/yanyiwu/husky.git
[limonp]:http://github.com/yanyiwu/limonp.git
[husky]:http://github.com/yanyiwu/husky.git
[issue50]:https://github.com/yanyiwu/cppjieba/issues/50
[qinwf]:https://github.com/yanyiwu/cppjieba/pull/53#issuecomment-176264929
[jieba]:https://github.com/fxsjy/jieba
[@questionfish]:https://github.com/questionfish

View File

@ -1,29 +1,31 @@
CMAKE_MINIMUM_REQUIRED (VERSION 3.10)
PROJECT(CPPJIEBA)
CMAKE_MINIMUM_REQUIRED (VERSION 2.6)
INCLUDE_DIRECTORIES(${PROJECT_SOURCE_DIR}/deps/limonp/include
${PROJECT_SOURCE_DIR}/include)
if (CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
set (CMAKE_INSTALL_PREFIX "/usr/local/cppjieba" CACHE PATH "default install path" FORCE )
if(NOT DEFINED CMAKE_CXX_STANDARD)
set(CMAKE_CXX_STANDARD 11)
endif()
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
ADD_DEFINITIONS(-O3 -g)
# Define a variable to check if this is the top-level project
if(NOT DEFINED CPPJIEBA_TOP_LEVEL_PROJECT)
if(CMAKE_CURRENT_SOURCE_DIR STREQUAL CMAKE_SOURCE_DIR)
set(CPPJIEBA_TOP_LEVEL_PROJECT ON)
else()
set(CPPJIEBA_TOP_LEVEL_PROJECT OFF)
endif()
endif()
ADD_DEFINITIONS(-O3 -Wall -g)
IF(APPLE) # mac os
ADD_DEFINITIONS(-std=c++0x)
endif()
if(CPPJIEBA_TOP_LEVEL_PROJECT)
ENABLE_TESTING()
# cmake .. -DENC=GBK
# IF (DEFINED ENC)
# ADD_DEFINITIONS(-DCPPJIEBA_${ENC})
# ENDIF()
ADD_SUBDIRECTORY(server)
ADD_SUBDIRECTORY(dict)
ADD_SUBDIRECTORY(script)
ADD_SUBDIRECTORY(conf)
ADD_SUBDIRECTORY(test)
ENABLE_TESTING()
ADD_TEST(NAME ./test/test.run COMMAND ./test/test.run)
ADD_TEST(NAME ./load_test COMMAND ./load_test)
ADD_TEST(NAME ./demo COMMAND ./demo)
message(STATUS "MSVC value: ${MSVC}")
ADD_SUBDIRECTORY(test)
ADD_TEST(NAME ./test/test.run COMMAND ./test/test.run)
ADD_TEST(NAME ./load_test COMMAND ./load_test)
endif()

View File

@ -1,11 +0,0 @@
FROM ubuntu:14.04
MAINTAINER yanyiwu <i@yanyiwu.com>
RUN apt-get update
RUN apt-get install -y g++ cmake git
RUN git clone https://github.com/yanyiwu/cppjieba.git
RUN mkdir cppjieba/build
WORKDIR /cppjieba/build
RUN cmake ..
RUN make
EXPOSE 11200
CMD ["./bin/cjserver", "../test/testdata/server.conf"]

20
LICENSE Normal file
View File

@ -0,0 +1,20 @@
The MIT License (MIT)
Copyright (c) 2013
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

268
README.md
View File

@ -1,47 +1,49 @@
# CppJieba [![Build Status](https://travis-ci.org/yanyiwu/cppjieba.png?branch=master)](https://travis-ci.org/yanyiwu/cppjieba)
# CppJieba
[![CMake](https://github.com/yanyiwu/cppjieba/actions/workflows/cmake.yml/badge.svg)](https://github.com/yanyiwu/cppjieba/actions/workflows/cmake.yml)
[![Author](https://img.shields.io/badge/author-@yanyiwu-blue.svg?style=flat)](http://yanyiwu.com/)
[![Platform](https://img.shields.io/badge/platform-Linux,macOS,Windows-green.svg?style=flat)](https://github.com/yanyiwu/cppjieba)
[![Performance](https://img.shields.io/badge/performance-excellent-brightgreen.svg?style=flat)](http://yanyiwu.com/work/2015/06/14/jieba-series-performance-test.html)
[![Tag](https://img.shields.io/github/v/tag/yanyiwu/cppjieba.svg)](https://github.com/yanyiwu/cppjieba/releases)
## 简介
CppJieba是"结巴(Jieba)"中文分词的C++版本
代码细节详解请见 [代码详解]
### 主要特点
## 特性
- 🚀 高性能:经过线上环境验证的稳定性和性能表现
- 📦 易集成:源代码以头文件形式提供 (`include/cppjieba/*.hpp`),包含即可使用
- 🔍 多种分词模式:支持精确模式、全模式、搜索引擎模式等
- 📚 自定义词典:支持用户自定义词典,支持多词典路径(使用'|'或';'分隔)
- 💻 跨平台:支持 Linux、macOS、Windows 操作系统
- 🌈 UTF-8编码原生支持 UTF-8 编码的中文处理
+ 源代码都写进头文件`src/*.hpp`里,`include`即可使用。
+ 支持`utf-8, gbk`编码,但是推荐使用`utf-8`编码, 因为`gbk`编码缺少严格测试,慎用。
+ 内置分词服务`server/server.cpp`在linux环境下可安装使用(可选)可通过http参数选择不同分词算法进行分词。
+ 项目自带较为完善的单元测试,核心功能中文分词(utf8)的稳定性接受过线上环境检验。
+ 支持载自定义用户词典,多路径时支持分隔符'|'或者';'分隔。
+ 支持 `linux` , `mac osx` 操作系统。
+ 支持 `Docker`
+ 提供 C语言 api接口调用 [cjieba]。
## 快速开始
## 用法
### 环境要求
### 依赖软件
- C++ 编译器:
- g++ (推荐 4.1 以上版本)
- 或 clang++
- cmake (推荐 2.6 以上版本)
* `g++ (version >= 4.1 recommended) or clang++`;
* `cmake (version >= 2.6 recommended)`;
### 下载和编译
### 安装步骤
```sh
git clone --depth=10 --branch=master git://github.com/yanyiwu/cppjieba.git
git clone https://github.com/yanyiwu/cppjieba.git
cd cppjieba
git submodule init
git submodule update
mkdir build
cd build
cmake ..
make
```
有兴趣的可以跑跑测试(可选):
```
make test
```
## Demo
## 使用示例
```
./demo
@ -51,134 +53,29 @@ make test
```
[demo] Cut With HMM
我/是/拖拉机/学院/手扶拖拉机/专业/的/。/不用/多久//我/就/会/升职/加薪//当上/CEO//走上/人生/巅峰/。
他/来到/了/网易/杭研/大厦
[demo] Cut Without HMM
我/是/拖拉机/学院/手扶拖拉机/专业/的/。/不用/多久//我/就/会/升职/加薪//当/上/C/E/O//走上/人生/巅峰/。
他/来到/了/网易/杭/研/大厦
我来到北京清华大学
[demo] CutAll
我/是/拖拉/拖拉机/学院/手扶/手扶拖拉机/拖拉/拖拉机/专业/的/。/不用/多久//我/就/会升/升职/加薪//当上/C/E/O//走上/人生/巅峰/。[demo] CutForSearch
我/是/拖拉机/学院/手扶/手扶拖拉机/拖拉/拖拉机/专业/的/。/不用/多久//我/就/会/升职/加薪//当上/CEO//走上/人生/巅峰/。
我/来到/北京/清华/清华大学/华大/大学
小明硕士毕业于中国科学院计算所,后在日本京都大学深造
[demo] CutForSearch
小明/硕士/毕业/于/中国/科学/学院/科学院/中国科学院/计算/计算所//后/在/日本/京都/大学/日本京都大学/深造
[demo] Insert User Word
男默/女泪
男默女泪
[demo] Locate Words
南京市, 0, 3
长江大桥, 3, 7
[demo] TAGGING
[demo] CutForSearch Word With Offset
[{"word": "小明", "offset": 0}, {"word": "硕士", "offset": 6}, {"word": "毕业", "offset": 12}, {"word": "于", "offset": 18}, {"word": "中国", "offset": 21}, {"word": "科学", "offset": 27}, {"word": "学院", "offset": 30}, {"word": "科学院", "offset": 27}, {"word": "中国科学院", "offset": 21}, {"word": "计算", "offset": 36}, {"word": "计算所", "offset": 36}, {"word": "", "offset": 45}, {"word": "后", "offset": 48}, {"word": "在", "offset": 51}, {"word": "日本", "offset": 54}, {"word": "京都", "offset": 60}, {"word": "大学", "offset": 66}, {"word": "日本京都大学", "offset": 54}, {"word": "深造", "offset": 72}]
[demo] Tagging
我是拖拉机学院手扶拖拉机专业的。不用多久我就会升职加薪当上CEO走上人生巅峰。
["我:r", "是:v", "拖拉机:n", "学院:n", "手扶拖拉机:n", "专业:n", "的:uj", "。:x", "不用:v", "多久:m", ":x", "我:r", "就:d", "会:v", "升职:v", "加薪:nr", ":x", "当上:t", "CEO:eng", ":x", "走上:v", "人生:n", "巅峰:n", "。:x"]
[我:r, 是:v, 拖拉机:n, 学院:n, 手扶拖拉机:n, 专业:n, 的:uj, 。:x, 不用:v, 多久:m, :x, 我:r, 就:d, 会:v, 升职:v, 加薪:nr, :x, 当上:t, CEO:eng, :x, 走上:v, 人生:n, 巅峰:n, 。:x]
[demo] Keyword Extraction
我是拖拉机学院手扶拖拉机专业的。不用多久我就会升职加薪当上CEO走上人生巅峰。
[{"word": "CEO", "offset": [93], "weight": 11.7392}, {"word": "升职", "offset": [72], "weight": 10.8562}, {"word": "加薪", "offset": [78], "weight": 10.6426}, {"word": "手扶拖拉机", "offset": [21], "weight": 10.0089}, {"word": "巅峰", "offset": [111], "weight": 9.49396}]
```
详细请看 `test/demo.cpp`.
## 服务使用
服务默认使用 MixSegment 切词方式,如果想要修改成其他方式,请参考 `server/server.cpp` 源码文件。
将对应的方式的代码行注释去掉,重新编译即可。
### 启动服务
```
./bin/cjserver ../conf/server_example.conf
```
### 客户端请求示例
```
curl "http://127.0.0.1:11200/?key=南京市长江大桥"
```
```
["南京市", "长江大桥"]
```
```
curl "http://127.0.0.1:11200/?key=南京市长江大桥&format=simple"
```
```
南京市 长江大桥
```
默认切词算法是MixSegment切词算法如果想要使用其他算法切词可以使用参数method来设置。
示例如下:
```
curl "http://127.0.0.1:11200/?key=南京市长江大桥&format=simple&method=MP"
curl "http://127.0.0.1:11200/?key=南京市长江大桥&format=simple&method=HMM"
curl "http://127.0.0.1:11200/?key=南京市长江大桥&format=simple&method=MIX"
curl "http://127.0.0.1:11200/?key=南京市长江大桥&format=simple&method=FULL"
curl "http://127.0.0.1:11200/?key=南京市长江大桥&format=simple&method=QUERY"
```
用 chrome 浏览器打开也行 ( chrome 设置默认编码是`utf-8`):
同时也支持HTTP POST模式使用如下调用:
```
curl -d "南京市长江大桥" "http://127.0.0.1:11200/"
```
返回结果如下:
```
["南京市", "长江大桥"]
```
因为 HTTP GET 请求有长度限制如果需要请求长文的请使用POST请求。
### 安装服务(仅限 linux 系统)
如果有需要**安装使用**的,可以按照如下操作:
```
sudo make install
```
### 服务启动和停止(仅限 linux 系统)
```
cd /usr/local/cppjieba
./script/cjserver.start
./script/cjserver.stop
```
### 卸载服务(仅限 linux 系统)
```sh
rm -rf /usr/local/cppjieba
```
## Docker 示例
安装和启动
```
sudo docker pull yanyiwu/cppjieba
sudo docker run -d -P yanyiwu/cppjieba
```
```
sudo docker ps
```
```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7c29325e9c20 yanyiwu/cppjieba:latest "./bin/cjserver ../t 4 minutes ago Up 4 minutes 0.0.0.0:49160->11200/tcp angry_wilson
```
可以看到正在运行的 Docker 容器(容器内运行着 `cjserver` 服务),并且服务的端口号被映射为 `0.0.0.0:49160`
所以现在可以来一发测试了:
```
curl "http://0.0.0.0:49160/?key=南京市长江大桥"
```
预期结果如下:
```
["南京市", "长江大桥"]
```
For more details, please see [demo](https://github.com/yanyiwu/cppjieba-demo).
### 分词结果示例
@ -284,7 +181,7 @@ Query方法先使用Mix方法切词对于切出来的较长的词再使用Ful
["CEO:11.7392", "升职:10.8562", "加薪:10.6426", "手扶拖拉机:10.0089", "巅峰:9.49396"]
```
详细请见 `test/demo.cpp`.
For more details, please see [demo](https://github.com/yanyiwu/cppjieba-demo).
### 词性标注
@ -293,7 +190,7 @@ Query方法先使用Mix方法切词对于切出来的较长的词再使用Ful
["我:r", "是:v", "拖拉机:n", "学院:n", "手扶拖拉机:n", "专业:n", "的:uj", "。:x", "不用:v", "多久:m", ":x", "我:r", "就:d", "会:v", "升职:v", "加薪:nr", ":x", "当上:t", "CEO:eng", ":x", "走上:v", "人生:n", "巅峰:n", "。:x"]
```
详细请看 `test/demo.cpp`.
For more details, please see [demo](https://github.com/yanyiwu/cppjieba-demo).
支持自定义词性。
比如在(`dict/user.dict.utf8`)增加一行
@ -310,72 +207,39 @@ Query方法先使用Mix方法切词对于切出来的较长的词再使用Ful
## 其它词典资料分享
+ [dict.367W.utf8.tar.gz] iLife(`562193561@qq.com`)
+ [dict.367W.utf8] iLife(562193561 at qq.com)
## 应用
## 生态系统
+ [GoJieba] go语言版本的结巴中文分词。
+ [cppjiebapy] 由 [jannson] 开发的供 python 模块调用的项目 [cppjiebapy], 相关讨论 [cppjiebapy_discussion] .
+ [NodeJieba] Node.js 版本的结巴中文分词。
+ [simhash] 中文文档的的相似度计算
+ [exjieba] Erlang 版本的结巴中文分词。
+ [jiebaR] R语言版本的结巴中文分词。
+ [libcppjieba] 是最简单易懂的CppJieba头文件库使用示例库。
+ [KeywordServer] 50行搭建一个中文关键词抽取服务。
+ [cjieba] C语言版本的结巴分词。
+ [jieba_rb] Ruby 版本的结巴分词。
+ [iosjieba] iOS 版本的结巴分词。
+ [pg_jieba] PostgreSQL 数据库的分词插件。
+ [ngx_http_cppjieba_module] Nginx 分词插件。
+ [gitbook-plugin-search-pro] 支持中文搜索的 gitbook 插件。
CppJieba 已经被广泛应用于各种编程语言的分词实现中:
## 线上演示
- [GoJieba](https://github.com/yanyiwu/gojieba) - Go 语言版本
- [NodeJieba](https://github.com/yanyiwu/nodejieba) - Node.js 版本
- [CJieba](https://github.com/yanyiwu/cjieba) - C 语言版本
- [jiebaR](https://github.com/qinwf/jiebaR) - R 语言版本
- [exjieba](https://github.com/falood/exjieba) - Erlang 版本
- [jieba_rb](https://github.com/altkatz/jieba_rb) - Ruby 版本
- [iosjieba](https://github.com/yanyiwu/iosjieba) - iOS 版本
- [phpjieba](https://github.com/jonnywang/phpjieba) - PHP 版本
- [perl5-jieba](https://metacpan.org/pod/distribution/Lingua-ZH-Jieba/lib/Lingua/ZH/Jieba.pod) - Perl 版本
http://cppjieba-webdemo.herokuapp.com/
(建议使用chrome打开)
### 应用项目
## 性能评测
- [simhash](https://github.com/yanyiwu/simhash) - 中文文档相似度计算
- [pg_jieba](https://github.com/jaiminpan/pg_jieba) - PostgreSQL 分词插件
- [gitbook-plugin-search-pro](https://plugins.gitbook.com/plugin/search-pro) - Gitbook 中文搜索插件
- [ngx_http_cppjieba_module](https://github.com/yanyiwu/ngx_http_cppjieba_module) - Nginx 分词插件
[Jieba中文分词系列性能评测]
## 贡献指南
## 客服
我们欢迎各种形式的贡献,包括但不限于:
+ Email: `i@yanyiwu.com`
+ QQ: 64162451
- 提交问题和建议
- 改进文档
- 提交代码修复
- 添加新功能
![image](http://7viirv.com1.z0.glb.clouddn.com/5a7d1b5c0d_yanyiwu_personal_qrcodes.jpg)
## 鸣谢
如果您觉得 CppJieba 对您有帮助,欢迎 star ⭐️ 支持项目!
"结巴"中文分词作者: SunJunyi https://github.com/fxsjy/jieba
## 许可证
MIT http://yanyiwu.mit-license.org
## 作者
- yanyiwu https://github.com/yanyiwu i@yanyiwu.com
- aholic https://github.com/aholic ruochen.xu@gmail.com
[GoJieba]:https://github.com/yanyiwu/gojieba
[CppJieba]:https://github.com/yanyiwu/cppjieba
[jannson]:https://github.com/jannson
[cppjiebapy]:https://github.com/jannson/cppjiebapy
[cppjiebapy_discussion]:https://github.com/yanyiwu/cppjieba/issues/1
[NodeJieba]:https://github.com/yanyiwu/nodejieba
[jiebaR]:https://github.com/qinwf/jiebaR
[simhash]:https://github.com/yanyiwu/simhash
[代码详解]:https://github.com/yanyiwu/cppjieba/wiki/CppJieba%E4%BB%A3%E7%A0%81%E8%AF%A6%E8%A7%A3
[libcppjieba]:https://github.com/yanyiwu/libcppjieba
[issue25]:https://github.com/yanyiwu/cppjieba/issues/25
[exjieba]:https://github.com/falood/exjieba
[KeywordServer]:https://github.com/yanyiwu/keyword_server
[ngx_http_cppjieba_module]:https://github.com/yanyiwu/ngx_http_cppjieba_module
[dict.367W.utf8.tar.gz]:http://pan.baidu.com/s/1o6A0BWY
[cjieba]:http://github.com/yanyiwu/cjieba
[jieba_rb]:https://github.com/altkatz/jieba_rb
[iosjieba]:https://github.com/yanyiwu/iosjieba
[Jieba中文分词系列性能评测]:http://yanyiwu.com/work/2015/06/14/jieba-series-performance-test.html
[pg_jieba]:https://github.com/jaiminpan/pg_jieba
[gitbook-plugin-search-pro]:https://plugins.gitbook.com/plugin/search-pro

View File

@ -1 +0,0 @@
INSTALL(FILES server.conf DESTINATION conf)

View File

@ -1,20 +0,0 @@
# config
#socket listen port
port=11200
thread_number=4
queue_max_size=4096
#dict path
dict_path=/usr/local/cppjieba/dict/jieba.dict.utf8
#model path
model_path=/usr/local/cppjieba/dict/hmm_model.utf8
#user_dict_path
user_dict_path=/usr/local/cppjieba/dict/user.dict.utf8
idf_path=/usr/local/cppjieba/dict/idf.utf8
stop_words_path=/usr/local/cppjieba/dict/stop_words.utf8

View File

@ -1,19 +0,0 @@
# config
#socket listen port
port=11200
thread_number=4
queue_max_size=4096
#dict path
dict_path=../dict/jieba.dict.utf8
#model path
model_path=../dict/hmm_model.utf8
user_dict_path=../dict/user.dict.utf8
idf_path=../dict/idf.utf8
stop_words_path=../dict/stop_words.utf8

1
deps/limonp vendored Submodule

@ -0,0 +1 @@
Subproject commit 5c82a3f17e4e0adc6a5decfe245054b0ed533d1a

View File

@ -1,9 +0,0 @@
INSTALL(FILES
hmm_model.utf8
jieba.dict.utf8
user.dict.utf8
idf.utf8
stop_words.utf8
DESTINATION
dict
)

View File

@ -312698,7 +312698,6 @@ T恤 4 n
部属 1126 n
部属工作 3 n
部属院校 3 n
部手机 33 n
部族 643 n
部标 4 n
部省级 2 n

View File

@ -1,3 +1,4 @@
云计算
韩玉鉴赏
蓝翔 nz
区块链 10 nz

View File

@ -0,0 +1,280 @@
#ifndef CPPJIEBA_DICT_TRIE_HPP
#define CPPJIEBA_DICT_TRIE_HPP
#include <algorithm>
#include <fstream>
#include <cstring>
#include <cstdlib>
#include <cmath>
#include <deque>
#include <set>
#include <string>
#include <unordered_set>
#include "limonp/StringUtil.hpp"
#include "limonp/Logging.hpp"
#include "Unicode.hpp"
#include "Trie.hpp"
namespace cppjieba {
const double MIN_DOUBLE = -3.14e+100;
const double MAX_DOUBLE = 3.14e+100;
const size_t DICT_COLUMN_NUM = 3;
const char* const UNKNOWN_TAG = "";
class DictTrie {
public:
enum UserWordWeightOption {
WordWeightMin,
WordWeightMedian,
WordWeightMax,
}; // enum UserWordWeightOption
DictTrie(const std::string& dict_path, const std::string& user_dict_paths = "", UserWordWeightOption user_word_weight_opt = WordWeightMedian) {
Init(dict_path, user_dict_paths, user_word_weight_opt);
}
~DictTrie() {
delete trie_;
}
bool InsertUserWord(const std::string& word, const std::string& tag = UNKNOWN_TAG) {
DictUnit node_info;
if (!MakeNodeInfo(node_info, word, user_word_default_weight_, tag)) {
return false;
}
active_node_infos_.push_back(node_info);
trie_->InsertNode(node_info.word, &active_node_infos_.back());
return true;
}
bool InsertUserWord(const std::string& word,int freq, const std::string& tag = UNKNOWN_TAG) {
DictUnit node_info;
double weight = freq ? log(1.0 * freq / freq_sum_) : user_word_default_weight_ ;
if (!MakeNodeInfo(node_info, word, weight , tag)) {
return false;
}
active_node_infos_.push_back(node_info);
trie_->InsertNode(node_info.word, &active_node_infos_.back());
return true;
}
bool DeleteUserWord(const std::string& word, const std::string& tag = UNKNOWN_TAG) {
DictUnit node_info;
if (!MakeNodeInfo(node_info, word, user_word_default_weight_, tag)) {
return false;
}
trie_->DeleteNode(node_info.word, &node_info);
return true;
}
const DictUnit* Find(RuneStrArray::const_iterator begin, RuneStrArray::const_iterator end) const {
return trie_->Find(begin, end);
}
void Find(RuneStrArray::const_iterator begin,
RuneStrArray::const_iterator end,
std::vector<struct Dag>&res,
size_t max_word_len = MAX_WORD_LENGTH) const {
trie_->Find(begin, end, res, max_word_len);
}
bool Find(const std::string& word)
{
const DictUnit *tmp = NULL;
RuneStrArray runes;
if (!DecodeUTF8RunesInString(word, runes))
{
XLOG(ERROR) << "Decode failed.";
}
tmp = Find(runes.begin(), runes.end());
if (tmp == NULL)
{
return false;
}
else
{
return true;
}
}
bool IsUserDictSingleChineseWord(const Rune& word) const {
return IsIn(user_dict_single_chinese_word_, word);
}
double GetMinWeight() const {
return min_weight_;
}
void InserUserDictNode(const std::string& line) {
std::vector<std::string> buf;
DictUnit node_info;
limonp::Split(line, buf, " ");
if(buf.size() == 1){
MakeNodeInfo(node_info,
buf[0],
user_word_default_weight_,
UNKNOWN_TAG);
} else if (buf.size() == 2) {
MakeNodeInfo(node_info,
buf[0],
user_word_default_weight_,
buf[1]);
} else if (buf.size() == 3) {
int freq = atoi(buf[1].c_str());
assert(freq_sum_ > 0.0);
double weight = log(1.0 * freq / freq_sum_);
MakeNodeInfo(node_info, buf[0], weight, buf[2]);
}
static_node_infos_.push_back(node_info);
if (node_info.word.size() == 1) {
user_dict_single_chinese_word_.insert(node_info.word[0]);
}
}
void LoadUserDict(const std::vector<std::string>& buf) {
for (size_t i = 0; i < buf.size(); i++) {
InserUserDictNode(buf[i]);
}
}
void LoadUserDict(const std::set<std::string>& buf) {
std::set<std::string>::const_iterator iter;
for (iter = buf.begin(); iter != buf.end(); iter++){
InserUserDictNode(*iter);
}
}
void LoadUserDict(const std::string& filePaths) {
std::vector<std::string> files = limonp::Split(filePaths, "|;");
for (size_t i = 0; i < files.size(); i++) {
std::ifstream ifs(files[i].c_str());
XCHECK(ifs.is_open()) << "open " << files[i] << " failed";
std::string line;
while(getline(ifs, line)) {
if (line.size() == 0) {
continue;
}
InserUserDictNode(line);
}
}
}
private:
void Init(const std::string& dict_path, const std::string& user_dict_paths, UserWordWeightOption user_word_weight_opt) {
LoadDict(dict_path);
freq_sum_ = CalcFreqSum(static_node_infos_);
CalculateWeight(static_node_infos_, freq_sum_);
SetStaticWordWeights(user_word_weight_opt);
if (user_dict_paths.size()) {
LoadUserDict(user_dict_paths);
}
Shrink(static_node_infos_);
CreateTrie(static_node_infos_);
}
void CreateTrie(const std::vector<DictUnit>& dictUnits) {
assert(dictUnits.size());
std::vector<Unicode> words;
std::vector<const DictUnit*> valuePointers;
for (size_t i = 0 ; i < dictUnits.size(); i ++) {
words.push_back(dictUnits[i].word);
valuePointers.push_back(&dictUnits[i]);
}
trie_ = new Trie(words, valuePointers);
}
bool MakeNodeInfo(DictUnit& node_info,
const std::string& word,
double weight,
const std::string& tag) {
if (!DecodeUTF8RunesInString(word, node_info.word)) {
XLOG(ERROR) << "UTF-8 decode failed for dict word: " << word;
return false;
}
node_info.weight = weight;
node_info.tag = tag;
return true;
}
void LoadDict(const std::string& filePath) {
std::ifstream ifs(filePath.c_str());
XCHECK(ifs.is_open()) << "open " << filePath << " failed.";
std::string line;
std::vector<std::string> buf;
DictUnit node_info;
while (getline(ifs, line)) {
limonp::Split(line, buf, " ");
XCHECK(buf.size() == DICT_COLUMN_NUM) << "split result illegal, line:" << line;
MakeNodeInfo(node_info,
buf[0],
atof(buf[1].c_str()),
buf[2]);
static_node_infos_.push_back(node_info);
}
}
static bool WeightCompare(const DictUnit& lhs, const DictUnit& rhs) {
return lhs.weight < rhs.weight;
}
void SetStaticWordWeights(UserWordWeightOption option) {
XCHECK(!static_node_infos_.empty());
std::vector<DictUnit> x = static_node_infos_;
std::sort(x.begin(), x.end(), WeightCompare);
min_weight_ = x[0].weight;
max_weight_ = x[x.size() - 1].weight;
median_weight_ = x[x.size() / 2].weight;
switch (option) {
case WordWeightMin:
user_word_default_weight_ = min_weight_;
break;
case WordWeightMedian:
user_word_default_weight_ = median_weight_;
break;
default:
user_word_default_weight_ = max_weight_;
break;
}
}
double CalcFreqSum(const std::vector<DictUnit>& node_infos) const {
double sum = 0.0;
for (size_t i = 0; i < node_infos.size(); i++) {
sum += node_infos[i].weight;
}
return sum;
}
void CalculateWeight(std::vector<DictUnit>& node_infos, double sum) const {
assert(sum > 0.0);
for (size_t i = 0; i < node_infos.size(); i++) {
DictUnit& node_info = node_infos[i];
assert(node_info.weight > 0.0);
node_info.weight = log(double(node_info.weight)/sum);
}
}
void Shrink(std::vector<DictUnit>& units) const {
std::vector<DictUnit>(units.begin(), units.end()).swap(units);
}
std::vector<DictUnit> static_node_infos_;
std::deque<DictUnit> active_node_infos_; // must not be std::vector
Trie * trie_;
double freq_sum_;
double min_weight_;
double max_weight_;
double median_weight_;
double user_word_default_weight_;
std::unordered_set<Rune> user_dict_single_chinese_word_;
};
}
#endif

View File

@ -4,10 +4,10 @@
#include <algorithm>
#include <set>
#include <cassert>
#include "limonp/Logger.hpp"
#include "limonp/Logging.hpp"
#include "DictTrie.hpp"
#include "SegmentBase.hpp"
#include "TransCode.hpp"
#include "Unicode.hpp"
namespace cppjieba {
class FullSegment: public SegmentBase {
@ -15,7 +15,6 @@ class FullSegment: public SegmentBase {
FullSegment(const string& dictPath) {
dictTrie_ = new DictTrie(dictPath);
isNeedDestroy_ = true;
LogInfo("FullSegment init %s ok", dictPath.c_str());
}
FullSegment(const DictTrie* dictTrie)
: dictTrie_(dictTrie), isNeedDestroy_(false) {
@ -28,44 +27,56 @@ class FullSegment: public SegmentBase {
}
void Cut(const string& sentence,
vector<string>& words) const {
vector<Word> tmp;
Cut(sentence, tmp);
GetStringsFromWords(tmp, words);
}
void Cut(const string& sentence,
vector<Word>& words) const {
PreFilter pre_filter(symbols_, sentence);
PreFilter::Range range;
vector<Unicode> uwords;
uwords.reserve(sentence.size());
vector<WordRange> wrs;
wrs.reserve(sentence.size()/2);
while (pre_filter.HasNext()) {
range = pre_filter.Next();
Cut(range.begin, range.end, uwords);
Cut(range.begin, range.end, wrs);
}
TransCode::Encode(uwords, words);
words.clear();
words.reserve(wrs.size());
GetWordsFromWordRanges(sentence, wrs, words);
}
void Cut(Unicode::const_iterator begin,
Unicode::const_iterator end,
vector<Unicode>& res) const {
//resut of searching in trie tree
void Cut(RuneStrArray::const_iterator begin,
RuneStrArray::const_iterator end,
vector<WordRange>& res) const {
// result of searching in trie tree
LocalVector<pair<size_t, const DictUnit*> > tRes;
//max index of res's words
int maxIdx = 0;
// max index of res's words
size_t maxIdx = 0;
// always equals to (uItr - begin)
int uIdx = 0;
size_t uIdx = 0;
//tmp variables
int wordLen = 0;
// tmp variables
size_t wordLen = 0;
assert(dictTrie_);
vector<struct Dag> dags;
dictTrie_->Find(begin, end, dags);
for (size_t i = 0; i < dags.size(); i++) {
for (size_t j = 0; j < dags[i].nexts.size(); j++) {
size_t nextoffset = dags[i].nexts[j].first;
assert(nextoffset < dags.size());
const DictUnit* du = dags[i].nexts[j].second;
if (du == NULL) {
if (dags[i].nexts.size() == 1 && maxIdx <= uIdx) {
res.push_back(Unicode(1, dags[i].rune));
WordRange wr(begin + i, begin + nextoffset);
res.push_back(wr);
}
} else {
wordLen = du->word.size();
if (wordLen >= 2 || (dags[i].nexts.size() == 1 && maxIdx <= uIdx)) {
res.push_back(du->word);
WordRange wr(begin + i, begin + nextoffset);
res.push_back(wr);
}
}
maxIdx = uIdx + wordLen > maxIdx ? uIdx + wordLen : maxIdx;

View File

@ -2,11 +2,12 @@
#define CPPJIEBA_HMMMODEL_H
#include "limonp/StringUtil.hpp"
#include "Trie.hpp"
namespace cppjieba {
using namespace limonp;
typedef unordered_map<uint16_t, double> EmitProbMap;
typedef unordered_map<Rune, double> EmitProbMap;
struct HMMModel {
/*
@ -32,59 +33,45 @@ struct HMMModel {
}
void LoadModel(const string& filePath) {
ifstream ifile(filePath.c_str());
if (!ifile.is_open()) {
LogFatal("open %s failed.", filePath.c_str());
}
XCHECK(ifile.is_open()) << "open " << filePath << " failed";
string line;
vector<string> tmp;
vector<string> tmp2;
//Load startProb
if (!GetLine(ifile, line)) {
LogFatal("Load startProb");
}
split(line, tmp, " ");
if (tmp.size() != STATUS_SUM) {
LogFatal("start_p illegal");
}
XCHECK(GetLine(ifile, line));
Split(line, tmp, " ");
XCHECK(tmp.size() == STATUS_SUM);
for (size_t j = 0; j< tmp.size(); j++) {
startProb[j] = atof(tmp[j].c_str());
}
//Load transProb
for (size_t i = 0; i < STATUS_SUM; i++) {
if (!GetLine(ifile, line)) {
LogFatal("Load transProb failed.");
}
split(line, tmp, " ");
if (tmp.size() != STATUS_SUM) {
LogFatal("trans_p illegal");
}
XCHECK(GetLine(ifile, line));
Split(line, tmp, " ");
XCHECK(tmp.size() == STATUS_SUM);
for (size_t j =0; j < STATUS_SUM; j++) {
transProb[i][j] = atof(tmp[j].c_str());
}
}
//Load emitProbB
if (!GetLine(ifile, line) || !LoadEmitProb(line, emitProbB)) {
LogFatal("Load emitProbB failed.");
}
XCHECK(GetLine(ifile, line));
XCHECK(LoadEmitProb(line, emitProbB));
//Load emitProbE
if (!GetLine(ifile, line) || !LoadEmitProb(line, emitProbE)) {
LogFatal("Load emitProbE failed.");
}
XCHECK(GetLine(ifile, line));
XCHECK(LoadEmitProb(line, emitProbE));
//Load emitProbM
if (!GetLine(ifile, line) || !LoadEmitProb(line, emitProbM)) {
LogFatal("Load emitProbM failed.");
}
XCHECK(GetLine(ifile, line));
XCHECK(LoadEmitProb(line, emitProbM));
//Load emitProbS
if (!GetLine(ifile, line) || !LoadEmitProb(line, emitProbS)) {
LogFatal("Load emitProbS failed.");
}
XCHECK(GetLine(ifile, line));
XCHECK(LoadEmitProb(line, emitProbS));
}
double GetEmitProb(const EmitProbMap* ptMp, uint16_t key,
double GetEmitProb(const EmitProbMap* ptMp, Rune key,
double defVal)const {
EmitProbMap::const_iterator cit = ptMp->find(key);
if (cit == ptMp->end()) {
@ -94,11 +81,11 @@ struct HMMModel {
}
bool GetLine(ifstream& ifile, string& line) {
while (getline(ifile, line)) {
trim(line);
Trim(line);
if (line.empty()) {
continue;
}
if (startsWith(line, "#")) {
if (StartsWith(line, "#")) {
continue;
}
return true;
@ -111,15 +98,15 @@ struct HMMModel {
}
vector<string> tmp, tmp2;
Unicode unicode;
split(line, tmp, ",");
Split(line, tmp, ",");
for (size_t i = 0; i < tmp.size(); i++) {
split(tmp[i], tmp2, ":");
Split(tmp[i], tmp2, ":");
if (2 != tmp2.size()) {
LogError("emitProb illegal.");
XLOG(ERROR) << "emitProb illegal.";
return false;
}
if (!TransCode::Decode(tmp2[0], unicode) || unicode.size() != 1) {
LogError("TransCode failed.");
if (!DecodeUTF8RunesInString(tmp2[0], unicode) || unicode.size() != 1) {
XLOG(ERROR) << "TransCode failed.";
return false;
}
mp[unicode[0]] = atof(tmp2[1].c_str());

View File

@ -11,8 +11,8 @@
namespace cppjieba {
class HMMSegment: public SegmentBase {
public:
HMMSegment(const string& filePath) {
model_ = new HMMModel(filePath);
HMMSegment(const string& filePath)
: model_(new HMMModel(filePath)), isNeedDestroy_(true) {
}
HMMSegment(const HMMModel* model)
: model_(model), isNeedDestroy_(false) {
@ -25,21 +25,29 @@ class HMMSegment: public SegmentBase {
void Cut(const string& sentence,
vector<string>& words) const {
vector<Word> tmp;
Cut(sentence, tmp);
GetStringsFromWords(tmp, words);
}
void Cut(const string& sentence,
vector<Word>& words) const {
PreFilter pre_filter(symbols_, sentence);
PreFilter::Range range;
vector<Unicode> uwords;
uwords.reserve(sentence.size());
vector<WordRange> wrs;
wrs.reserve(sentence.size()/2);
while (pre_filter.HasNext()) {
range = pre_filter.Next();
Cut(range.begin, range.end, uwords);
Cut(range.begin, range.end, wrs);
}
TransCode::Encode(uwords, words);
words.clear();
words.reserve(wrs.size());
GetWordsFromWordRanges(sentence, wrs, words);
}
void Cut(Unicode::const_iterator begin, Unicode::const_iterator end, vector<Unicode>& res) const {
Unicode::const_iterator left = begin;
Unicode::const_iterator right = begin;
void Cut(RuneStrArray::const_iterator begin, RuneStrArray::const_iterator end, vector<WordRange>& res) const {
RuneStrArray::const_iterator left = begin;
RuneStrArray::const_iterator right = begin;
while (right != end) {
if (*right < 0x80) {
if (right->rune < 0x80) {
if (left != right) {
InternalCut(left, right, res);
}
@ -55,7 +63,8 @@ class HMMSegment: public SegmentBase {
}
right ++;
} while (false);
res.push_back(Unicode(left, right));
WordRange wr(left, right - 1);
res.push_back(wr);
left = right;
} else {
right++;
@ -67,15 +76,15 @@ class HMMSegment: public SegmentBase {
}
private:
// sequential letters rule
Unicode::const_iterator SequentialLetterRule(Unicode::const_iterator begin, Unicode::const_iterator end) const {
Rune x = *begin;
RuneStrArray::const_iterator SequentialLetterRule(RuneStrArray::const_iterator begin, RuneStrArray::const_iterator end) const {
Rune x = begin->rune;
if (('a' <= x && x <= 'z') || ('A' <= x && x <= 'Z')) {
begin ++;
} else {
return begin;
}
while (begin != end) {
x = *begin;
x = begin->rune;
if (('a' <= x && x <= 'z') || ('A' <= x && x <= 'Z') || ('0' <= x && x <= '9')) {
begin ++;
} else {
@ -85,15 +94,15 @@ class HMMSegment: public SegmentBase {
return begin;
}
//
Unicode::const_iterator NumbersRule(Unicode::const_iterator begin, Unicode::const_iterator end) const {
Rune x = *begin;
RuneStrArray::const_iterator NumbersRule(RuneStrArray::const_iterator begin, RuneStrArray::const_iterator end) const {
Rune x = begin->rune;
if ('0' <= x && x <= '9') {
begin ++;
} else {
return begin;
}
while (begin != end) {
x = *begin;
x = begin->rune;
if ( ('0' <= x && x <= '9') || x == '.') {
begin++;
} else {
@ -102,23 +111,24 @@ class HMMSegment: public SegmentBase {
}
return begin;
}
void InternalCut(Unicode::const_iterator begin, Unicode::const_iterator end, vector<Unicode>& res) const {
void InternalCut(RuneStrArray::const_iterator begin, RuneStrArray::const_iterator end, vector<WordRange>& res) const {
vector<size_t> status;
Viterbi(begin, end, status);
Unicode::const_iterator left = begin;
Unicode::const_iterator right;
RuneStrArray::const_iterator left = begin;
RuneStrArray::const_iterator right;
for (size_t i = 0; i < status.size(); i++) {
if (status[i] % 2) { //if (HMMModel::E == status[i] || HMMModel::S == status[i])
right = begin + i + 1;
res.push_back(Unicode(left, right));
WordRange wr(left, right - 1);
res.push_back(wr);
left = right;
}
}
}
void Viterbi(Unicode::const_iterator begin,
Unicode::const_iterator end,
void Viterbi(RuneStrArray::const_iterator begin,
RuneStrArray::const_iterator end,
vector<size_t>& status) const {
size_t Y = HMMModel::STATUS_SUM;
size_t X = end - begin;
@ -132,7 +142,7 @@ class HMMSegment: public SegmentBase {
//start
for (size_t y = 0; y < Y; y++) {
weight[0 + y * X] = model_->startProb[y] + model_->GetEmitProb(model_->emitProbVec[y], *begin, MIN_DOUBLE);
weight[0 + y * X] = model_->startProb[y] + model_->GetEmitProb(model_->emitProbVec[y], begin->rune, MIN_DOUBLE);
path[0 + y * X] = -1;
}
@ -143,7 +153,7 @@ class HMMSegment: public SegmentBase {
now = x + y*X;
weight[now] = MIN_DOUBLE;
path[now] = HMMModel::E; // warning
emitProb = model_->GetEmitProb(model_->emitProbVec[y], *(begin+x), MIN_DOUBLE);
emitProb = model_->GetEmitProb(model_->emitProbVec[y], (begin+x)->rune, MIN_DOUBLE);
for (size_t preY = 0; preY < Y; preY++) {
old = x - 1 + preY * X;
tmp = weight[old] + model_->transProb[preY][y] + emitProb;

169
include/cppjieba/Jieba.hpp Normal file
View File

@ -0,0 +1,169 @@
#ifndef CPPJIEAB_JIEBA_H
#define CPPJIEAB_JIEBA_H
#include "QuerySegment.hpp"
#include "KeywordExtractor.hpp"
namespace cppjieba {
class Jieba {
public:
Jieba(const string& dict_path = "",
const string& model_path = "",
const string& user_dict_path = "",
const string& idf_path = "",
const string& stop_word_path = "")
: dict_trie_(getPath(dict_path, "jieba.dict.utf8"), getPath(user_dict_path, "user.dict.utf8")),
model_(getPath(model_path, "hmm_model.utf8")),
mp_seg_(&dict_trie_),
hmm_seg_(&model_),
mix_seg_(&dict_trie_, &model_),
full_seg_(&dict_trie_),
query_seg_(&dict_trie_, &model_),
extractor(&dict_trie_, &model_,
getPath(idf_path, "idf.utf8"),
getPath(stop_word_path, "stop_words.utf8")) {
}
~Jieba() {
}
struct LocWord {
string word;
size_t begin;
size_t end;
}; // struct LocWord
void Cut(const string& sentence, vector<string>& words, bool hmm = true) const {
mix_seg_.Cut(sentence, words, hmm);
}
void Cut(const string& sentence, vector<Word>& words, bool hmm = true) const {
mix_seg_.Cut(sentence, words, hmm);
}
void CutAll(const string& sentence, vector<string>& words) const {
full_seg_.Cut(sentence, words);
}
void CutAll(const string& sentence, vector<Word>& words) const {
full_seg_.Cut(sentence, words);
}
void CutForSearch(const string& sentence, vector<string>& words, bool hmm = true) const {
query_seg_.Cut(sentence, words, hmm);
}
void CutForSearch(const string& sentence, vector<Word>& words, bool hmm = true) const {
query_seg_.Cut(sentence, words, hmm);
}
void CutHMM(const string& sentence, vector<string>& words) const {
hmm_seg_.Cut(sentence, words);
}
void CutHMM(const string& sentence, vector<Word>& words) const {
hmm_seg_.Cut(sentence, words);
}
void CutSmall(const string& sentence, vector<string>& words, size_t max_word_len) const {
mp_seg_.Cut(sentence, words, max_word_len);
}
void CutSmall(const string& sentence, vector<Word>& words, size_t max_word_len) const {
mp_seg_.Cut(sentence, words, max_word_len);
}
void Tag(const string& sentence, vector<pair<string, string> >& words) const {
mix_seg_.Tag(sentence, words);
}
string LookupTag(const string &str) const {
return mix_seg_.LookupTag(str);
}
bool InsertUserWord(const string& word, const string& tag = UNKNOWN_TAG) {
return dict_trie_.InsertUserWord(word, tag);
}
bool InsertUserWord(const string& word,int freq, const string& tag = UNKNOWN_TAG) {
return dict_trie_.InsertUserWord(word,freq, tag);
}
bool DeleteUserWord(const string& word, const string& tag = UNKNOWN_TAG) {
return dict_trie_.DeleteUserWord(word, tag);
}
bool Find(const string& word)
{
return dict_trie_.Find(word);
}
void ResetSeparators(const string& s) {
//TODO
mp_seg_.ResetSeparators(s);
hmm_seg_.ResetSeparators(s);
mix_seg_.ResetSeparators(s);
full_seg_.ResetSeparators(s);
query_seg_.ResetSeparators(s);
}
const DictTrie* GetDictTrie() const {
return &dict_trie_;
}
const HMMModel* GetHMMModel() const {
return &model_;
}
void LoadUserDict(const vector<string>& buf) {
dict_trie_.LoadUserDict(buf);
}
void LoadUserDict(const set<string>& buf) {
dict_trie_.LoadUserDict(buf);
}
void LoadUserDict(const string& path) {
dict_trie_.LoadUserDict(path);
}
private:
static string pathJoin(const string& dir, const string& filename) {
if (dir.empty()) {
return filename;
}
char last_char = dir[dir.length() - 1];
if (last_char == '/' || last_char == '\\') {
return dir + filename;
} else {
#ifdef _WIN32
return dir + '\\' + filename;
#else
return dir + '/' + filename;
#endif
}
}
static string getCurrentDirectory() {
string path(__FILE__);
size_t pos = path.find_last_of("/\\");
return (pos == string::npos) ? "" : path.substr(0, pos);
}
static string getPath(const string& path, const string& default_file) {
if (path.empty()) {
string current_dir = getCurrentDirectory();
string parent_dir = current_dir.substr(0, current_dir.find_last_of("/\\"));
string grandparent_dir = parent_dir.substr(0, parent_dir.find_last_of("/\\"));
return pathJoin(pathJoin(grandparent_dir, "dict"), default_file);
}
return path;
}
DictTrie dict_trie_;
HMMModel model_;
// They share the same dict trie and model
MPSegment mp_seg_;
HMMSegment hmm_seg_;
MixSegment mix_seg_;
FullSegment full_seg_;
QuerySegment query_seg_;
public:
KeywordExtractor extractor;
}; // class Jieba
} // namespace cppjieba
#endif // CPPJIEAB_JIEBA_H

View File

@ -0,0 +1,149 @@
#ifndef CPPJIEBA_KEYWORD_EXTRACTOR_H
#define CPPJIEBA_KEYWORD_EXTRACTOR_H
#include <algorithm>
#include <unordered_map>
#include <unordered_set>
#include "MixSegment.hpp"
namespace cppjieba {
/*utf8*/
class KeywordExtractor {
public:
struct Word {
std::string word;
std::vector<size_t> offsets;
double weight;
}; // struct Word
KeywordExtractor(const std::string& dictPath,
const std::string& hmmFilePath,
const std::string& idfPath,
const std::string& stopWordPath,
const std::string& userDict = "")
: segment_(dictPath, hmmFilePath, userDict) {
LoadIdfDict(idfPath);
LoadStopWordDict(stopWordPath);
}
KeywordExtractor(const DictTrie* dictTrie,
const HMMModel* model,
const std::string& idfPath,
const std::string& stopWordPath)
: segment_(dictTrie, model) {
LoadIdfDict(idfPath);
LoadStopWordDict(stopWordPath);
}
~KeywordExtractor() {
}
void Extract(const std::string& sentence, std::vector<std::string>& keywords, size_t topN) const {
std::vector<Word> topWords;
Extract(sentence, topWords, topN);
for (size_t i = 0; i < topWords.size(); i++) {
keywords.push_back(topWords[i].word);
}
}
void Extract(const std::string& sentence, std::vector<pair<std::string, double> >& keywords, size_t topN) const {
std::vector<Word> topWords;
Extract(sentence, topWords, topN);
for (size_t i = 0; i < topWords.size(); i++) {
keywords.push_back(pair<std::string, double>(topWords[i].word, topWords[i].weight));
}
}
void Extract(const std::string& sentence, std::vector<Word>& keywords, size_t topN) const {
std::vector<std::string> words;
segment_.Cut(sentence, words);
std::map<std::string, Word> wordmap;
size_t offset = 0;
for (size_t i = 0; i < words.size(); ++i) {
size_t t = offset;
offset += words[i].size();
if (IsSingleWord(words[i]) || stopWords_.find(words[i]) != stopWords_.end()) {
continue;
}
wordmap[words[i]].offsets.push_back(t);
wordmap[words[i]].weight += 1.0;
}
if (offset != sentence.size()) {
XLOG(ERROR) << "words illegal";
return;
}
keywords.clear();
keywords.reserve(wordmap.size());
for (std::map<std::string, Word>::iterator itr = wordmap.begin(); itr != wordmap.end(); ++itr) {
std::unordered_map<std::string, double>::const_iterator cit = idfMap_.find(itr->first);
if (cit != idfMap_.end()) {
itr->second.weight *= cit->second;
} else {
itr->second.weight *= idfAverage_;
}
itr->second.word = itr->first;
keywords.push_back(itr->second);
}
topN = min(topN, keywords.size());
std::partial_sort(keywords.begin(), keywords.begin() + topN, keywords.end(), Compare);
keywords.resize(topN);
}
private:
void LoadIdfDict(const std::string& idfPath) {
std::ifstream ifs(idfPath.c_str());
XCHECK(ifs.is_open()) << "open " << idfPath << " failed";
std::string line ;
std::vector<std::string> buf;
double idf = 0.0;
double idfSum = 0.0;
size_t lineno = 0;
for (; getline(ifs, line); lineno++) {
buf.clear();
if (line.empty()) {
XLOG(ERROR) << "lineno: " << lineno << " empty. skipped.";
continue;
}
limonp::Split(line, buf, " ");
if (buf.size() != 2) {
XLOG(ERROR) << "line: " << line << ", lineno: " << lineno << " empty. skipped.";
continue;
}
idf = atof(buf[1].c_str());
idfMap_[buf[0]] = idf;
idfSum += idf;
}
assert(lineno);
idfAverage_ = idfSum / lineno;
assert(idfAverage_ > 0.0);
}
void LoadStopWordDict(const std::string& filePath) {
std::ifstream ifs(filePath.c_str());
XCHECK(ifs.is_open()) << "open " << filePath << " failed";
std::string line ;
while (getline(ifs, line)) {
stopWords_.insert(line);
}
assert(stopWords_.size());
}
static bool Compare(const Word& lhs, const Word& rhs) {
return lhs.weight > rhs.weight;
}
MixSegment segment_;
std::unordered_map<std::string, double> idfMap_;
double idfAverage_;
std::unordered_set<std::string> stopWords_;
}; // class KeywordExtractor
inline std::ostream& operator << (std::ostream& os, const KeywordExtractor::Word& word) {
return os << "{\"word\": \"" << word.word << "\", \"offset\": " << word.offsets << ", \"weight\": " << word.weight << "}";
}
} // namespace cppjieba
#endif

View File

@ -4,18 +4,17 @@
#include <algorithm>
#include <set>
#include <cassert>
#include "limonp/Logger.hpp"
#include "limonp/Logging.hpp"
#include "DictTrie.hpp"
#include "SegmentBase.hpp"
#include "SegmentTagged.hpp"
#include "PosTagger.hpp"
namespace cppjieba {
class MPSegment: public SegmentBase {
class MPSegment: public SegmentTagged {
public:
MPSegment(const string& dictPath, const string& userDictPath = "") {
dictTrie_ = new DictTrie(dictPath, userDictPath);
isNeedDestroy_ = true;
LogInfo("MPSegment init(%s) ok", dictPath.c_str());
MPSegment(const string& dictPath, const string& userDictPath = "")
: dictTrie_(new DictTrie(dictPath, userDictPath)), isNeedDestroy_(true) {
}
MPSegment(const DictTrie* dictTrie)
: dictTrie_(dictTrie), isNeedDestroy_(false) {
@ -27,22 +26,35 @@ class MPSegment: public SegmentBase {
}
}
void Cut(const string& sentence, vector<string>& words) const {
Cut(sentence, words, MAX_WORD_LENGTH);
}
void Cut(const string& sentence,
vector<string>& words,
size_t max_word_len) const {
vector<Word> tmp;
Cut(sentence, tmp, max_word_len);
GetStringsFromWords(tmp, words);
}
void Cut(const string& sentence,
vector<string>& words,
vector<Word>& words,
size_t max_word_len = MAX_WORD_LENGTH) const {
PreFilter pre_filter(symbols_, sentence);
PreFilter::Range range;
vector<Unicode> uwords;
uwords.reserve(sentence.size());
vector<WordRange> wrs;
wrs.reserve(sentence.size()/2);
while (pre_filter.HasNext()) {
range = pre_filter.Next();
Cut(range.begin, range.end, uwords, max_word_len);
Cut(range.begin, range.end, wrs, max_word_len);
}
TransCode::Encode(uwords, words);
words.clear();
words.reserve(wrs.size());
GetWordsFromWordRanges(sentence, wrs, words);
}
void Cut(Unicode::const_iterator begin,
Unicode::const_iterator end,
vector<Unicode>& words,
void Cut(RuneStrArray::const_iterator begin,
RuneStrArray::const_iterator end,
vector<WordRange>& words,
size_t max_word_len = MAX_WORD_LENGTH) const {
vector<Dag> dags;
dictTrie_->Find(begin,
@ -50,13 +62,17 @@ class MPSegment: public SegmentBase {
dags,
max_word_len);
CalcDP(dags);
CutByDag(dags, words);
CutByDag(begin, end, dags, words);
}
const DictTrie* GetDictTrie() const {
return dictTrie_;
}
bool Tag(const string& src, vector<pair<string, string> >& res) const {
return tagger_.Tag(src, res, *this);
}
bool IsUserDictSingleChineseWord(const Rune& value) const {
return dictTrie_->IsUserDictSingleChineseWord(value);
}
@ -90,16 +106,21 @@ class MPSegment: public SegmentBase {
}
}
}
void CutByDag(const vector<Dag>& dags,
vector<Unicode>& words) const {
void CutByDag(RuneStrArray::const_iterator begin,
RuneStrArray::const_iterator end,
const vector<Dag>& dags,
vector<WordRange>& words) const {
size_t i = 0;
while (i < dags.size()) {
const DictUnit* p = dags[i].pInfo;
if (p) {
words.push_back(p->word);
assert(p->word.size() >= 1);
WordRange wr(begin + i, begin + i + p->word.size() - 1);
words.push_back(wr);
i += p->word.size();
} else { //single chinese word
words.push_back(Unicode(1, dags[i].rune));
WordRange wr(begin + i, begin + i);
words.push_back(wr);
i++;
}
}
@ -107,6 +128,8 @@ class MPSegment: public SegmentBase {
const DictTrie* dictTrie_;
bool isNeedDestroy_;
PosTagger tagger_;
}; // class MPSegment
} // namespace cppjieba

View File

@ -5,15 +5,15 @@
#include "MPSegment.hpp"
#include "HMMSegment.hpp"
#include "limonp/StringUtil.hpp"
#include "PosTagger.hpp"
namespace cppjieba {
class MixSegment: public SegmentBase {
class MixSegment: public SegmentTagged {
public:
MixSegment(const string& mpSegDict, const string& hmmSegDict,
const string& userDict = "")
: mpSeg_(mpSegDict, userDict),
hmmSeg_(hmmSegDict) {
LogInfo("MixSegment init %s, %s", mpSegDict.c_str(), hmmSegDict.c_str());
}
MixSegment(const DictTrie* dictTrie, const HMMModel* model)
: mpSeg_(dictTrie), hmmSeg_(model) {
@ -21,55 +21,63 @@ class MixSegment: public SegmentBase {
~MixSegment() {
}
void Cut(const string& sentence, vector<string>& words, bool hmm = true) const {
void Cut(const string& sentence, vector<string>& words) const {
Cut(sentence, words, true);
}
void Cut(const string& sentence, vector<string>& words, bool hmm) const {
vector<Word> tmp;
Cut(sentence, tmp, hmm);
GetStringsFromWords(tmp, words);
}
void Cut(const string& sentence, vector<Word>& words, bool hmm = true) const {
PreFilter pre_filter(symbols_, sentence);
PreFilter::Range range;
vector<Unicode> uwords;
uwords.reserve(sentence.size());
vector<WordRange> wrs;
wrs.reserve(sentence.size() / 2);
while (pre_filter.HasNext()) {
range = pre_filter.Next();
Cut(range.begin, range.end, uwords, hmm);
Cut(range.begin, range.end, wrs, hmm);
}
TransCode::Encode(uwords, words);
words.clear();
words.reserve(wrs.size());
GetWordsFromWordRanges(sentence, wrs, words);
}
void Cut(Unicode::const_iterator begin, Unicode::const_iterator end, vector<Unicode>& res, bool hmm) const {
void Cut(RuneStrArray::const_iterator begin, RuneStrArray::const_iterator end, vector<WordRange>& res, bool hmm) const {
if (!hmm) {
mpSeg_.Cut(begin, end, res);
return;
}
vector<Unicode> words;
vector<WordRange> words;
assert(end >= begin);
words.reserve(end - begin);
mpSeg_.Cut(begin, end, words);
vector<Unicode> hmmRes;
vector<WordRange> hmmRes;
hmmRes.reserve(end - begin);
Unicode piece;
piece.reserve(end - begin);
for (size_t i = 0, j = 0; i < words.size(); i++) {
for (size_t i = 0; i < words.size(); i++) {
//if mp Get a word, it's ok, put it into result
if (1 != words[i].size() || (words[i].size() == 1 && mpSeg_.IsUserDictSingleChineseWord(words[i][0]))) {
if (words[i].left != words[i].right || (words[i].left == words[i].right && mpSeg_.IsUserDictSingleChineseWord(words[i].left->rune))) {
res.push_back(words[i]);
continue;
}
// if mp Get a single one and it is not in userdict, collect it in sequence
j = i;
while (j < words.size() && 1 == words[j].size() && !mpSeg_.IsUserDictSingleChineseWord(words[j][0])) {
piece.push_back(words[j][0]);
size_t j = i;
while (j < words.size() && words[j].left == words[j].right && !mpSeg_.IsUserDictSingleChineseWord(words[j].left->rune)) {
j++;
}
// Cut the sequence with hmm
hmmSeg_.Cut(piece.begin(), piece.end(), hmmRes);
assert(j - 1 >= i);
// TODO
hmmSeg_.Cut(words[i].left, words[j - 1].left + 1, hmmRes);
//put hmm result to result
for (size_t k = 0; k < hmmRes.size(); k++) {
res.push_back(hmmRes[k]);
}
//clear tmp vars
piece.clear();
hmmRes.clear();
//let i jump over this piece
@ -80,9 +88,19 @@ class MixSegment: public SegmentBase {
const DictTrie* GetDictTrie() const {
return mpSeg_.GetDictTrie();
}
bool Tag(const string& src, vector<pair<string, string> >& res) const {
return tagger_.Tag(src, res, *this);
}
string LookupTag(const string &str) const {
return tagger_.LookupTag(str, *this);
}
private:
MPSegment mpSeg_;
HMMSegment hmmSeg_;
PosTagger tagger_;
}; // class MixSegment

View File

@ -1,8 +1,8 @@
#ifndef CPPJIEBA_POS_TAGGING_H
#define CPPJIEBA_POS_TAGGING_H
#include "MixSegment.hpp"
#include "limonp/StringUtil.hpp"
#include "SegmentTagged.hpp"
#include "DictTrie.hpp"
namespace cppjieba {
@ -14,47 +14,46 @@ static const char* const POS_X = "x";
class PosTagger {
public:
PosTagger(const string& dictPath,
const string& hmmFilePath,
const string& userDictPath = "")
: segment_(dictPath, hmmFilePath, userDictPath) {
}
PosTagger(const DictTrie* dictTrie, const HMMModel* model)
: segment_(dictTrie, model) {
PosTagger() {
}
~PosTagger() {
}
bool Tag(const string& src, vector<pair<string, string> >& res) const {
bool Tag(const string& src, vector<pair<string, string> >& res, const SegmentTagged& segment) const {
vector<string> CutRes;
segment_.Cut(src, CutRes);
segment.Cut(src, CutRes);
const DictUnit *tmp = NULL;
Unicode unico;
const DictTrie * dict = segment_.GetDictTrie();
assert(dict != NULL);
for (vector<string>::iterator itr = CutRes.begin(); itr != CutRes.end(); ++itr) {
if (!TransCode::Decode(*itr, unico)) {
LogError("Decode failed.");
return false;
}
tmp = dict->Find(unico.begin(), unico.end());
if (tmp == NULL || tmp->tag.empty()) {
res.push_back(make_pair(*itr, SpecialRule(unico)));
} else {
res.push_back(make_pair(*itr, tmp->tag));
}
res.push_back(make_pair(*itr, LookupTag(*itr, segment)));
}
return !res.empty();
}
string LookupTag(const string &str, const SegmentTagged& segment) const {
const DictUnit *tmp = NULL;
RuneStrArray runes;
const DictTrie * dict = segment.GetDictTrie();
assert(dict != NULL);
if (!DecodeUTF8RunesInString(str, runes)) {
XLOG(ERROR) << "UTF-8 decode failed for word: " << str;
return POS_X;
}
tmp = dict->Find(runes.begin(), runes.end());
if (tmp == NULL || tmp->tag.empty()) {
return SpecialRule(runes);
} else {
return tmp->tag;
}
}
private:
const char* SpecialRule(const Unicode& unicode) const {
const char* SpecialRule(const RuneStrArray& unicode) const {
size_t m = 0;
size_t eng = 0;
for (size_t i = 0; i < unicode.size() && eng < unicode.size() / 2; i++) {
if (unicode[i] < 0x80) {
if (unicode[i].rune < 0x80) {
eng ++;
if ('0' <= unicode[i] && unicode[i] <= '9') {
if ('0' <= unicode[i].rune && unicode[i].rune <= '9') {
m++;
}
}
@ -71,7 +70,6 @@ class PosTagger {
return POS_ENG;
}
MixSegment segment_;
}; // class PosTagger
} // namespace cppjieba

View File

@ -1,32 +1,25 @@
#ifndef CPPJIEBA_PRE_FILTER_H
#define CPPJIEBA_PRE_FILTER_H
#include "TransCode.hpp"
#include "Trie.hpp"
#include "limonp/Logging.hpp"
namespace cppjieba {
//class PreFilterIterator {
// public:
// PreFilterIterator() {
// }
// ~PreFilterIterator() {
// }
//
// private:
// const unordered_set<Rune>& specialSymbols_;
//}; // PreFilterIterator
class PreFilter {
public:
//TODO use WordRange instead of Range
struct Range {
Unicode::const_iterator begin;
Unicode::const_iterator end;
RuneStrArray::const_iterator begin;
RuneStrArray::const_iterator end;
}; // struct Range
PreFilter(const unordered_set<Rune>& symbols,
const string& sentence)
: symbols_(symbols) {
TransCode::Decode(sentence, sentence_);
if (!DecodeUTF8RunesInString(sentence, sentence_)) {
XLOG(ERROR) << "UTF-8 decode failed for input sentence";
}
cursor_ = sentence_.begin();
}
~PreFilter() {
@ -38,7 +31,7 @@ class PreFilter {
Range range;
range.begin = cursor_;
while (cursor_ != sentence_.end()) {
if (isIn(symbols_, *cursor_)) {
if (IsIn(symbols_, cursor_->rune)) {
if (range.begin == cursor_) {
cursor_ ++;
}
@ -51,8 +44,8 @@ class PreFilter {
return range;
}
private:
Unicode::const_iterator cursor_;
Unicode sentence_;
RuneStrArray::const_iterator cursor_;
RuneStrArray sentence_;
const unordered_set<Rune>& symbols_;
}; // class PreFilter

View File

@ -0,0 +1,89 @@
#ifndef CPPJIEBA_QUERYSEGMENT_H
#define CPPJIEBA_QUERYSEGMENT_H
#include <algorithm>
#include <set>
#include <cassert>
#include "limonp/Logging.hpp"
#include "DictTrie.hpp"
#include "SegmentBase.hpp"
#include "FullSegment.hpp"
#include "MixSegment.hpp"
#include "Unicode.hpp"
namespace cppjieba {
class QuerySegment: public SegmentBase {
public:
QuerySegment(const string& dict, const string& model, const string& userDict = "")
: mixSeg_(dict, model, userDict),
trie_(mixSeg_.GetDictTrie()) {
}
QuerySegment(const DictTrie* dictTrie, const HMMModel* model)
: mixSeg_(dictTrie, model), trie_(dictTrie) {
}
~QuerySegment() {
}
void Cut(const string& sentence, vector<string>& words) const {
Cut(sentence, words, true);
}
void Cut(const string& sentence, vector<string>& words, bool hmm) const {
vector<Word> tmp;
Cut(sentence, tmp, hmm);
GetStringsFromWords(tmp, words);
}
void Cut(const string& sentence, vector<Word>& words, bool hmm = true) const {
PreFilter pre_filter(symbols_, sentence);
PreFilter::Range range;
vector<WordRange> wrs;
wrs.reserve(sentence.size()/2);
while (pre_filter.HasNext()) {
range = pre_filter.Next();
Cut(range.begin, range.end, wrs, hmm);
}
words.clear();
words.reserve(wrs.size());
GetWordsFromWordRanges(sentence, wrs, words);
}
void Cut(RuneStrArray::const_iterator begin, RuneStrArray::const_iterator end, vector<WordRange>& res, bool hmm) const {
//use mix Cut first
vector<WordRange> mixRes;
mixSeg_.Cut(begin, end, mixRes, hmm);
vector<WordRange> fullRes;
for (vector<WordRange>::const_iterator mixResItr = mixRes.begin(); mixResItr != mixRes.end(); mixResItr++) {
if (mixResItr->Length() > 2) {
for (size_t i = 0; i + 1 < mixResItr->Length(); i++) {
WordRange wr(mixResItr->left + i, mixResItr->left + i + 1);
if (trie_->Find(wr.left, wr.right + 1) != NULL) {
res.push_back(wr);
}
}
}
if (mixResItr->Length() > 3) {
for (size_t i = 0; i + 2 < mixResItr->Length(); i++) {
WordRange wr(mixResItr->left + i, mixResItr->left + i + 2);
if (trie_->Find(wr.left, wr.right + 1) != NULL) {
res.push_back(wr);
}
}
}
res.push_back(*mixResItr);
}
}
private:
bool IsAllAscii(const Unicode& s) const {
for(size_t i = 0; i < s.size(); i++) {
if (s[i] >= 0x80) {
return false;
}
}
return true;
}
MixSegment mixSeg_;
const DictTrie* trie_;
}; // QuerySegment
} // namespace cppjieba
#endif

View File

@ -0,0 +1,46 @@
#ifndef CPPJIEBA_SEGMENTBASE_H
#define CPPJIEBA_SEGMENTBASE_H
#include "limonp/Logging.hpp"
#include "PreFilter.hpp"
#include <cassert>
namespace cppjieba {
const char* const SPECIAL_SEPARATORS = " \t\n\xEF\xBC\x8C\xE3\x80\x82";
using namespace limonp;
class SegmentBase {
public:
SegmentBase() {
XCHECK(ResetSeparators(SPECIAL_SEPARATORS));
}
virtual ~SegmentBase() {
}
virtual void Cut(const string& sentence, vector<string>& words) const = 0;
bool ResetSeparators(const string& s) {
symbols_.clear();
RuneStrArray runes;
if (!DecodeUTF8RunesInString(s, runes)) {
XLOG(ERROR) << "UTF-8 decode failed for separators: " << s;
return false;
}
for (size_t i = 0; i < runes.size(); i++) {
if (!symbols_.insert(runes[i].rune).second) {
XLOG(ERROR) << s.substr(runes[i].offset, runes[i].len) << " already exists";
return false;
}
}
return true;
}
protected:
unordered_set<Rune> symbols_;
}; // class SegmentBase
} // cppjieba
#endif

View File

@ -0,0 +1,23 @@
#ifndef CPPJIEBA_SEGMENTTAGGED_H
#define CPPJIEBA_SEGMENTTAGGED_H
#include "SegmentBase.hpp"
namespace cppjieba {
class SegmentTagged : public SegmentBase{
public:
SegmentTagged() {
}
virtual ~SegmentTagged() {
}
virtual bool Tag(const string& src, vector<pair<string, string> >& res) const = 0;
virtual const DictTrie* GetDictTrie() const = 0;
}; // class SegmentTagged
} // cppjieba
#endif

View File

@ -0,0 +1,190 @@
#ifndef CPPJIEBA_TEXTRANK_EXTRACTOR_H
#define CPPJIEBA_TEXTRANK_EXTRACTOR_H
#include <cmath>
#include "Jieba.hpp"
namespace cppjieba {
using namespace limonp;
using namespace std;
class TextRankExtractor {
public:
typedef struct _Word {string word;vector<size_t> offsets;double weight;} Word; // struct Word
private:
typedef std::map<string,Word> WordMap;
class WordGraph{
private:
typedef double Score;
typedef string Node;
typedef std::set<Node> NodeSet;
typedef std::map<Node,double> Edges;
typedef std::map<Node,Edges> Graph;
//typedef std::unordered_map<Node,double> Edges;
//typedef std::unordered_map<Node,Edges> Graph;
double d;
Graph graph;
NodeSet nodeSet;
public:
WordGraph(): d(0.85) {};
WordGraph(double in_d): d(in_d) {};
void addEdge(Node start,Node end,double weight){
Edges temp;
Edges::iterator gotEdges;
nodeSet.insert(start);
nodeSet.insert(end);
graph[start][end]+=weight;
graph[end][start]+=weight;
}
void rank(WordMap &ws,size_t rankTime=10){
WordMap outSum;
Score wsdef, min_rank, max_rank;
if( graph.size() == 0)
return;
wsdef = 1.0 / graph.size();
for(Graph::iterator edges=graph.begin();edges!=graph.end();++edges){
// edges->first start节点edge->first end节点edge->second 权重
ws[edges->first].word=edges->first;
ws[edges->first].weight=wsdef;
outSum[edges->first].weight=0;
for(Edges::iterator edge=edges->second.begin();edge!=edges->second.end();++edge){
outSum[edges->first].weight+=edge->second;
}
}
//sort(nodeSet.begin(),nodeSet.end()); 是否需要排序?
for( size_t i=0; i<rankTime; i++ ){
for(NodeSet::iterator node = nodeSet.begin(); node != nodeSet.end(); node++ ){
double s = 0;
for( Edges::iterator edge= graph[*node].begin(); edge != graph[*node].end(); edge++ )
// edge->first end节点edge->second 权重
s += edge->second / outSum[edge->first].weight * ws[edge->first].weight;
ws[*node].weight = (1 - d) + d * s;
}
}
min_rank=max_rank=ws.begin()->second.weight;
for(WordMap::iterator i = ws.begin(); i != ws.end(); i ++){
if( i->second.weight < min_rank ){
min_rank = i->second.weight;
}
if( i->second.weight > max_rank ){
max_rank = i->second.weight;
}
}
for(WordMap::iterator i = ws.begin(); i != ws.end(); i ++){
ws[i->first].weight = (i->second.weight - min_rank / 10.0) / (max_rank - min_rank / 10.0);
}
}
};
public:
TextRankExtractor(const string& dictPath,
const string& hmmFilePath,
const string& stopWordPath,
const string& userDict = "")
: segment_(dictPath, hmmFilePath, userDict) {
LoadStopWordDict(stopWordPath);
}
TextRankExtractor(const DictTrie* dictTrie,
const HMMModel* model,
const string& stopWordPath)
: segment_(dictTrie, model) {
LoadStopWordDict(stopWordPath);
}
TextRankExtractor(const Jieba& jieba, const string& stopWordPath) : segment_(jieba.GetDictTrie(), jieba.GetHMMModel()) {
LoadStopWordDict(stopWordPath);
}
~TextRankExtractor() {
}
void Extract(const string& sentence, vector<string>& keywords, size_t topN) const {
vector<Word> topWords;
Extract(sentence, topWords, topN);
for (size_t i = 0; i < topWords.size(); i++) {
keywords.push_back(topWords[i].word);
}
}
void Extract(const string& sentence, vector<pair<string, double> >& keywords, size_t topN) const {
vector<Word> topWords;
Extract(sentence, topWords, topN);
for (size_t i = 0; i < topWords.size(); i++) {
keywords.push_back(pair<string, double>(topWords[i].word, topWords[i].weight));
}
}
void Extract(const string& sentence, vector<Word>& keywords, size_t topN, size_t span=5,size_t rankTime=10) const {
vector<string> words;
segment_.Cut(sentence, words);
TextRankExtractor::WordGraph graph;
WordMap wordmap;
size_t offset = 0;
for(size_t i=0; i < words.size(); i++){
size_t t = offset;
offset += words[i].size();
if (IsSingleWord(words[i]) || stopWords_.find(words[i]) != stopWords_.end()) {
continue;
}
for(size_t j=i+1,skip=0;j<i+span+skip && j<words.size();j++){
if (IsSingleWord(words[j]) || stopWords_.find(words[j]) != stopWords_.end()) {
skip++;
continue;
}
graph.addEdge(words[i],words[j],1);
}
wordmap[words[i]].offsets.push_back(t);
}
if (offset != sentence.size()) {
XLOG(ERROR) << "words illegal";
return;
}
graph.rank(wordmap,rankTime);
keywords.clear();
keywords.reserve(wordmap.size());
for (WordMap::iterator itr = wordmap.begin(); itr != wordmap.end(); ++itr) {
keywords.push_back(itr->second);
}
topN = min(topN, keywords.size());
partial_sort(keywords.begin(), keywords.begin() + topN, keywords.end(), Compare);
keywords.resize(topN);
}
private:
void LoadStopWordDict(const string& filePath) {
ifstream ifs(filePath.c_str());
XCHECK(ifs.is_open()) << "open " << filePath << " failed";
string line ;
while (getline(ifs, line)) {
stopWords_.insert(line);
}
assert(stopWords_.size());
}
static bool Compare(const Word &x,const Word &y){
return x.weight > y.weight;
}
MixSegment segment_;
unordered_set<string> stopWords_;
}; // class TextRankExtractor
inline ostream& operator << (ostream& os, const TextRankExtractor::Word& word) {
return os << "{\"word\": \"" << word.word << "\", \"offset\": " << word.offsets << ", \"weight\": " << word.weight << "}";
}
} // namespace cppjieba
#endif

200
include/cppjieba/Trie.hpp Normal file
View File

@ -0,0 +1,200 @@
#ifndef CPPJIEBA_TRIE_HPP
#define CPPJIEBA_TRIE_HPP
#include <vector>
#include <queue>
#include "limonp/StdExtension.hpp"
#include "Unicode.hpp"
namespace cppjieba {
using namespace std;
const size_t MAX_WORD_LENGTH = 512;
struct DictUnit {
Unicode word;
double weight;
string tag;
}; // struct DictUnit
// for debugging
// inline ostream & operator << (ostream& os, const DictUnit& unit) {
// string s;
// s << unit.word;
// return os << StringFormat("%s %s %.3lf", s.c_str(), unit.tag.c_str(), unit.weight);
// }
struct Dag {
RuneStr runestr;
// [offset, nexts.first]
limonp::LocalVector<pair<size_t, const DictUnit*> > nexts;
const DictUnit * pInfo;
double weight;
size_t nextPos; // TODO
Dag():runestr(), pInfo(NULL), weight(0.0), nextPos(0) {
}
}; // struct Dag
typedef Rune TrieKey;
class TrieNode {
public :
TrieNode(): next(NULL), ptValue(NULL) {
}
public:
typedef unordered_map<TrieKey, TrieNode*> NextMap;
NextMap *next;
const DictUnit *ptValue;
};
class Trie {
public:
Trie(const vector<Unicode>& keys, const vector<const DictUnit*>& valuePointers)
: root_(new TrieNode) {
CreateTrie(keys, valuePointers);
}
~Trie() {
DeleteNode(root_);
}
const DictUnit* Find(RuneStrArray::const_iterator begin, RuneStrArray::const_iterator end) const {
if (begin == end) {
return NULL;
}
const TrieNode* ptNode = root_;
TrieNode::NextMap::const_iterator citer;
for (RuneStrArray::const_iterator it = begin; it != end; it++) {
if (NULL == ptNode->next) {
return NULL;
}
citer = ptNode->next->find(it->rune);
if (ptNode->next->end() == citer) {
return NULL;
}
ptNode = citer->second;
}
return ptNode->ptValue;
}
void Find(RuneStrArray::const_iterator begin,
RuneStrArray::const_iterator end,
vector<struct Dag>&res,
size_t max_word_len = MAX_WORD_LENGTH) const {
assert(root_ != NULL);
res.resize(end - begin);
const TrieNode *ptNode = NULL;
TrieNode::NextMap::const_iterator citer;
for (size_t i = 0; i < size_t(end - begin); i++) {
res[i].runestr = *(begin + i);
if (root_->next != NULL && root_->next->end() != (citer = root_->next->find(res[i].runestr.rune))) {
ptNode = citer->second;
} else {
ptNode = NULL;
}
if (ptNode != NULL) {
res[i].nexts.push_back(pair<size_t, const DictUnit*>(i, ptNode->ptValue));
} else {
res[i].nexts.push_back(pair<size_t, const DictUnit*>(i, static_cast<const DictUnit*>(NULL)));
}
for (size_t j = i + 1; j < size_t(end - begin) && (j - i + 1) <= max_word_len; j++) {
if (ptNode == NULL || ptNode->next == NULL) {
break;
}
citer = ptNode->next->find((begin + j)->rune);
if (ptNode->next->end() == citer) {
break;
}
ptNode = citer->second;
if (NULL != ptNode->ptValue) {
res[i].nexts.push_back(pair<size_t, const DictUnit*>(j, ptNode->ptValue));
}
}
}
}
void InsertNode(const Unicode& key, const DictUnit* ptValue) {
if (key.begin() == key.end()) {
return;
}
TrieNode::NextMap::const_iterator kmIter;
TrieNode *ptNode = root_;
for (Unicode::const_iterator citer = key.begin(); citer != key.end(); ++citer) {
if (NULL == ptNode->next) {
ptNode->next = new TrieNode::NextMap;
}
kmIter = ptNode->next->find(*citer);
if (ptNode->next->end() == kmIter) {
TrieNode *nextNode = new TrieNode;
ptNode->next->insert(make_pair(*citer, nextNode));
ptNode = nextNode;
} else {
ptNode = kmIter->second;
}
}
assert(ptNode != NULL);
ptNode->ptValue = ptValue;
}
void DeleteNode(const Unicode& key, const DictUnit* ptValue) {
if (key.begin() == key.end()) {
return;
}
//定义一个NextMap迭代器
TrieNode::NextMap::const_iterator kmIter;
//定义一个指向root的TrieNode指针
TrieNode *ptNode = root_;
for (Unicode::const_iterator citer = key.begin(); citer != key.end(); ++citer) {
//链表不存在元素
if (NULL == ptNode->next) {
return;
}
kmIter = ptNode->next->find(*citer);
//如果map中不存在,跳出循环
if (ptNode->next->end() == kmIter) {
break;
}
//从unordered_map中擦除该项
ptNode->next->erase(*citer);
//删除该node
ptNode = kmIter->second;
delete ptNode;
break;
}
return;
}
private:
void CreateTrie(const vector<Unicode>& keys, const vector<const DictUnit*>& valuePointers) {
if (valuePointers.empty() || keys.empty()) {
return;
}
assert(keys.size() == valuePointers.size());
for (size_t i = 0; i < keys.size(); i++) {
InsertNode(keys[i], valuePointers[i]);
}
}
void DeleteNode(TrieNode* node) {
if (NULL == node) {
return;
}
if (NULL != node->next) {
for (TrieNode::NextMap::iterator it = node->next->begin(); it != node->next->end(); ++it) {
DeleteNode(it->second);
}
delete node->next;
}
delete node;
}
TrieNode* root_;
}; // class Trie
} // namespace cppjieba
#endif // CPPJIEBA_TRIE_HPP

View File

@ -0,0 +1,227 @@
#ifndef CPPJIEBA_UNICODE_H
#define CPPJIEBA_UNICODE_H
#include <stdint.h>
#include <stdlib.h>
#include <string>
#include <vector>
#include <ostream>
#include "limonp/LocalVector.hpp"
namespace cppjieba {
using std::string;
using std::vector;
typedef uint32_t Rune;
struct Word {
string word;
uint32_t offset;
uint32_t unicode_offset;
uint32_t unicode_length;
Word(const string& w, uint32_t o)
: word(w), offset(o) {
}
Word(const string& w, uint32_t o, uint32_t unicode_offset, uint32_t unicode_length)
: word(w), offset(o), unicode_offset(unicode_offset), unicode_length(unicode_length) {
}
}; // struct Word
inline std::ostream& operator << (std::ostream& os, const Word& w) {
return os << "{\"word\": \"" << w.word << "\", \"offset\": " << w.offset << "}";
}
struct RuneStr {
Rune rune;
uint32_t offset;
uint32_t len;
uint32_t unicode_offset;
uint32_t unicode_length;
RuneStr(): rune(0), offset(0), len(0), unicode_offset(0), unicode_length(0) {
}
RuneStr(Rune r, uint32_t o, uint32_t l)
: rune(r), offset(o), len(l), unicode_offset(0), unicode_length(0) {
}
RuneStr(Rune r, uint32_t o, uint32_t l, uint32_t unicode_offset, uint32_t unicode_length)
: rune(r), offset(o), len(l), unicode_offset(unicode_offset), unicode_length(unicode_length) {
}
}; // struct RuneStr
inline std::ostream& operator << (std::ostream& os, const RuneStr& r) {
return os << "{\"rune\": \"" << r.rune << "\", \"offset\": " << r.offset << ", \"len\": " << r.len << "}";
}
typedef limonp::LocalVector<Rune> Unicode;
typedef limonp::LocalVector<struct RuneStr> RuneStrArray;
// [left, right]
struct WordRange {
RuneStrArray::const_iterator left;
RuneStrArray::const_iterator right;
WordRange(RuneStrArray::const_iterator l, RuneStrArray::const_iterator r)
: left(l), right(r) {
}
size_t Length() const {
return right - left + 1;
}
bool IsAllAscii() const {
for (RuneStrArray::const_iterator iter = left; iter <= right; ++iter) {
if (iter->rune >= 0x80) {
return false;
}
}
return true;
}
}; // struct WordRange
struct RuneStrLite {
uint32_t rune;
uint32_t len;
RuneStrLite(): rune(0), len(0) {
}
RuneStrLite(uint32_t r, uint32_t l): rune(r), len(l) {
}
}; // struct RuneStrLite
inline RuneStrLite DecodeUTF8ToRune(const char* str, size_t len) {
RuneStrLite rp(0, 0);
if (str == NULL || len == 0) {
return rp;
}
if (!(str[0] & 0x80)) { // 0xxxxxxx
// 7bit, total 7bit
rp.rune = (uint8_t)(str[0]) & 0x7f;
rp.len = 1;
} else if ((uint8_t)str[0] <= 0xdf && 1 < len) {
// 110xxxxxx
// 5bit, total 5bit
rp.rune = (uint8_t)(str[0]) & 0x1f;
// 6bit, total 11bit
rp.rune <<= 6;
rp.rune |= (uint8_t)(str[1]) & 0x3f;
rp.len = 2;
} else if((uint8_t)str[0] <= 0xef && 2 < len) { // 1110xxxxxx
// 4bit, total 4bit
rp.rune = (uint8_t)(str[0]) & 0x0f;
// 6bit, total 10bit
rp.rune <<= 6;
rp.rune |= (uint8_t)(str[1]) & 0x3f;
// 6bit, total 16bit
rp.rune <<= 6;
rp.rune |= (uint8_t)(str[2]) & 0x3f;
rp.len = 3;
} else if((uint8_t)str[0] <= 0xf7 && 3 < len) { // 11110xxxx
// 3bit, total 3bit
rp.rune = (uint8_t)(str[0]) & 0x07;
// 6bit, total 9bit
rp.rune <<= 6;
rp.rune |= (uint8_t)(str[1]) & 0x3f;
// 6bit, total 15bit
rp.rune <<= 6;
rp.rune |= (uint8_t)(str[2]) & 0x3f;
// 6bit, total 21bit
rp.rune <<= 6;
rp.rune |= (uint8_t)(str[3]) & 0x3f;
rp.len = 4;
} else {
rp.rune = 0;
rp.len = 0;
}
return rp;
}
inline bool DecodeUTF8RunesInString(const char* s, size_t len, RuneStrArray& runes) {
runes.clear();
runes.reserve(len / 2);
for (uint32_t i = 0, j = 0; i < len;) {
RuneStrLite rp = DecodeUTF8ToRune(s + i, len - i);
if (rp.len == 0) {
runes.clear();
return false;
}
RuneStr x(rp.rune, i, rp.len, j, 1);
runes.push_back(x);
i += rp.len;
++j;
}
return true;
}
inline bool DecodeUTF8RunesInString(const string& s, RuneStrArray& runes) {
return DecodeUTF8RunesInString(s.c_str(), s.size(), runes);
}
inline bool DecodeUTF8RunesInString(const char* s, size_t len, Unicode& unicode) {
unicode.clear();
RuneStrArray runes;
if (!DecodeUTF8RunesInString(s, len, runes)) {
return false;
}
unicode.reserve(runes.size());
for (size_t i = 0; i < runes.size(); i++) {
unicode.push_back(runes[i].rune);
}
return true;
}
inline bool IsSingleWord(const string& str) {
RuneStrLite rp = DecodeUTF8ToRune(str.c_str(), str.size());
return rp.len == str.size();
}
inline bool DecodeUTF8RunesInString(const string& s, Unicode& unicode) {
return DecodeUTF8RunesInString(s.c_str(), s.size(), unicode);
}
inline Unicode DecodeUTF8RunesInString(const string& s) {
Unicode result;
DecodeUTF8RunesInString(s, result);
return result;
}
// [left, right]
inline Word GetWordFromRunes(const string& s, RuneStrArray::const_iterator left, RuneStrArray::const_iterator right) {
assert(right->offset >= left->offset);
uint32_t len = right->offset - left->offset + right->len;
uint32_t unicode_length = right->unicode_offset - left->unicode_offset + right->unicode_length;
return Word(s.substr(left->offset, len), left->offset, left->unicode_offset, unicode_length);
}
inline string GetStringFromRunes(const string& s, RuneStrArray::const_iterator left, RuneStrArray::const_iterator right) {
assert(right->offset >= left->offset);
uint32_t len = right->offset - left->offset + right->len;
return s.substr(left->offset, len);
}
inline void GetWordsFromWordRanges(const string& s, const vector<WordRange>& wrs, vector<Word>& words) {
for (size_t i = 0; i < wrs.size(); i++) {
words.push_back(GetWordFromRunes(s, wrs[i].left, wrs[i].right));
}
}
inline vector<Word> GetWordsFromWordRanges(const string& s, const vector<WordRange>& wrs) {
vector<Word> result;
GetWordsFromWordRanges(s, wrs, result);
return result;
}
inline void GetStringsFromWords(const vector<Word>& words, vector<string>& strs) {
strs.resize(words.size());
for (size_t i = 0; i < words.size(); ++i) {
strs[i] = words[i].word;
}
}
} // namespace cppjieba
#endif // CPPJIEBA_UNICODE_H

View File

@ -1,6 +0,0 @@
INSTALL(PROGRAMS
cjserver.start
cjserver.stop
DESTINATION
script
)

View File

@ -1,12 +0,0 @@
#!/bin/sh
PATH=/usr/bin/:/usr/local/bin/:/sbin/:$PATH
PID=`pidof cjserver`
if [ ! -z "${PID}" ]
then
echo "please stop cjserver first."
else
/usr/local/cppjieba/bin/cjserver /usr/local/cppjieba/conf/server.conf >> /dev/null 2>&1 &
echo "service started."
fi

View File

@ -1,13 +0,0 @@
#!/bin/sh
PATH=/usr/bin/:/usr/local/bin/:/sbin/:$PATH
PID=`pidof cjserver`
if [ ! -z "${PID}" ]
then
kill ${PID}
sleep 1
echo "service stop ok."
else
echo "cjserver is not running."
fi

View File

@ -1,8 +0,0 @@
SET(EXECUTABLE_OUTPUT_PATH ${PROJECT_BINARY_DIR}/bin)
INCLUDE_DIRECTORIES(${PROJECT_SOURCE_DIR}/src)
ADD_EXECUTABLE(cjserver server.cpp)
TARGET_LINK_LIBRARIES(cjserver pthread)
INSTALL(TARGETS cjserver DESTINATION bin)

View File

@ -1,267 +0,0 @@
#ifndef HUSKY_HTTP_REQINFO_H
#define HUSKY_HTTP_REQINFO_H
#include <iostream>
#include <string>
#include "limonp/Logger.hpp"
#include "limonp/StringUtil.hpp"
namespace husky {
using namespace limonp;
using namespace std;
static const char* const KEY_METHOD = "METHOD";
static const char* const KEY_URI = "URI";
static const char* const KEY_PROTOCOL = "PROTOCOL";
typedef unsigned char BYTE;
inline BYTE toHex(BYTE x) {
return x > 9 ? x -10 + 'A': x + '0';
}
inline BYTE fromHex(BYTE x) {
return isdigit(x) ? x-'0' : x-'A'+10;
}
inline void URLEncode(const string &sIn, string& sOut) {
for( size_t ix = 0; ix < sIn.size(); ix++ ) {
BYTE buf[4];
memset( buf, 0, 4 );
if( isalnum( (BYTE)sIn[ix] ) ) {
buf[0] = sIn[ix];
} else {
buf[0] = '%';
buf[1] = toHex( (BYTE)sIn[ix] >> 4 );
buf[2] = toHex( (BYTE)sIn[ix] % 16);
}
sOut += (char *)buf;
}
};
inline void URLDecode(const string &sIn, string& sOut) {
for( size_t ix = 0; ix < sIn.size(); ix++ ) {
BYTE ch = 0;
if(sIn[ix]=='%') {
ch = (fromHex(sIn[ix+1])<<4);
ch |= fromHex(sIn[ix+2]);
ix += 2;
} else if(sIn[ix] == '+') {
ch = ' ';
} else {
ch = sIn[ix];
}
sOut += (char)ch;
}
}
class HttpReqInfo {
public:
HttpReqInfo() {
_isHeaderFinished = false;
_isBodyFinished = false;
_contentLength = 0;
}
bool parseHeader(const string& buffer) {
return parseHeader(buffer.c_str(), buffer.size());
}
bool parseHeader(const char* buffer, size_t len) {
string headerStr(buffer, len);
size_t lpos = 0, rpos = 0;
vector<string> buf;
rpos = headerStr.find("\n", lpos);
if(string::npos == rpos) {
LogError("headerStr[%s] illegal.", headerStr.c_str());
return false;
}
string firstline(headerStr, lpos, rpos - lpos);
trim(firstline);
split(firstline, buf, " ");
if (3 != buf.size()) {
LogError("parse header firstline[%s] failed.", firstline.c_str());
return false;
}
_headerMap[KEY_METHOD] = trim(buf[0]);
_headerMap[KEY_URI] = trim(buf[1]);
_headerMap[KEY_PROTOCOL] = trim(buf[2]);
_parseUri(_headerMap[KEY_URI], _path, _methodGetMap);
lpos = rpos + 1;
if(lpos >= headerStr.size()) {
LogError("headerStr[%s] illegal.", headerStr.c_str());
return false;
}
//message header begin
while(lpos < headerStr.size() && string::npos != (rpos = headerStr.find('\n', lpos)) && rpos > lpos) {
string s(headerStr, lpos, rpos - lpos);
size_t p = s.find(':');
if(string::npos == p) {
break;//encounter empty line
}
string k(s, 0, p);
string v(s, p+1);
trim(k);
trim(v);
if(k.empty()||v.empty()) {
LogError("headerStr[%s] illegal.", headerStr.c_str());
return false;
}
upper(k);
_headerMap[k] = v;
lpos = rpos + 1;
}
rpos ++;
_isHeaderFinished = true;
string content_length;
if(!find("CONTENT-LENGTH", content_length)) {
_isBodyFinished = true;
return true;
}
_contentLength = atoi(content_length.c_str());
if(rpos < headerStr.size()) {
appendBody(headerStr.c_str() + rpos, headerStr.size() - rpos);
}
return true;
//message header end
}
void appendBody(const char* buffer, size_t len) {
if(_isBodyFinished) {
return;
}
_body.append(buffer, len);
if(_body.size() >= _contentLength) {
_isBodyFinished = true;
} else {
_isBodyFinished = false;
}
}
bool isHeaderFinished() const {
return _isHeaderFinished;
}
bool isBodyFinished() const {
return _isBodyFinished;
}
const string& set(const string& key, const string& value) {
return _headerMap[key] = value;
}
bool find(const string& key, string& res)const {
return _find(_headerMap, key, res);
}
bool GET(const string& argKey, string& res)const {
string tmp;
if (!_find(_methodGetMap, argKey, tmp)) {
return false;
}
URLDecode(tmp, res);
return true;
}
bool GET(const string& argKey, int& res) const {
string tmp;
if (!GET(argKey, tmp)) {
return false;
}
res = atoi(tmp.c_str());
return true;
}
bool GET(const string& argKey, size_t& res) const {
int tmp = 0;
if (!GET(argKey, tmp) || tmp < 0) {
return false;
}
res = tmp;
return true;
}
//const string& getMethod() const
//{
// return _headerMap.find(KEY_METHOD)->second;
//}
bool isGET() const {
string str;
if(!_find(_headerMap, KEY_METHOD, str)) {
return false;
}
return str == "GET";
}
bool isPOST() const {
string str;
if(!_find(_headerMap, KEY_METHOD, str)) {
return false;
}
return str == "POST";
}
const unordered_map<string, string> & getMethodGetMap() const {
return _methodGetMap;
}
const unordered_map<string, string> & getHeaders() const {
return _headerMap;
}
const string& getBody() const {
return _body;
}
const string& getPath() const {
return _path;
}
private:
bool _isHeaderFinished;
bool _isBodyFinished;
size_t _contentLength;
unordered_map<string, string> _headerMap;
unordered_map<string, string> _methodGetMap;
string _path;
string _body;
friend ostream& operator<<(ostream& os, const HttpReqInfo& obj);
bool _find(const std::unordered_map<string, string>& mp, const string& key, string& res)const {
std::unordered_map<string, string>::const_iterator it = mp.find(key);
if(it == mp.end()) {
return false;
}
res = it->second;
return true;
}
void _parseUri(const string& uri, string& path, std::unordered_map<string, string>& mp) {
if(uri.empty()) {
return;
}
size_t pos = uri.find('?');
path = uri.substr(0, pos);
if(string::npos == pos) {
return ;
}
size_t kleft = 0, kright = 0;
size_t vleft = 0, vright = 0;
for(size_t i = pos + 1; i < uri.size();) {
kleft = i;
while(i < uri.size() && uri[i] != '=') {
i++;
}
if(i >= uri.size()) {
break;
}
kright = i;
i++;
vleft = i;
while(i < uri.size() && uri[i] != '&' && uri[i] != ' ') {
i++;
}
vright = i;
mp[uri.substr(kleft, kright - kleft)] = uri.substr(vleft, vright - vleft);
i++;
}
return;
}
};
inline std::ostream& operator << (std::ostream& os, const husky::HttpReqInfo& obj) {
return os << obj._headerMap << obj._methodGetMap/* << obj._methodPostMap*/ << obj._path << obj._body ;
}
}
#endif

View File

@ -1,17 +0,0 @@
#ifndef HUSKY_IREQUESTHANDLER_HPP
#define HUSKY_IREQUESTHANDLER_HPP
#include "HttpReqInfo.hpp"
namespace husky {
class IRequestHandler {
public:
virtual ~IRequestHandler() {
}
virtual bool doGET(const HttpReqInfo& httpReq, string& res) = 0;
virtual bool doPOST(const HttpReqInfo& httpReq, string& res) = 0;
};
}
#endif

View File

@ -1,56 +0,0 @@
#ifndef HUSKY_NET_UTILS_HPP
#define HUSKY_NET_UTILS_HPP
#include <stdio.h>
#include <string.h>
#include <cassert>
#include <sys/socket.h>
#include <sys/types.h>
#include <arpa/inet.h>
#include <stdlib.h>
#include <pthread.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <vector>
#include "limonp/StdExtension.hpp"
#include "limonp/Logger.hpp"
namespace husky {
static const size_t LISTEN_QUEUE_LEN = 1024;
typedef int SocketFd;
inline SocketFd CreateAndListenSocket(int port) {
SocketFd sock;
sock = socket(AF_INET, SOCK_STREAM, 0);
if (sock == -1) {
LogFatal("create socket failed");
}
int optval = 1; // nozero
if (-1 == setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof(optval))) {
LogFatal("setsockopt failed");
}
struct sockaddr_in addr;
addr.sin_family = AF_INET;
addr.sin_port = htons(port);
addr.sin_addr.s_addr = htonl(INADDR_ANY);
if (-1 == ::bind(sock, (sockaddr*)&addr, sizeof(addr))) {
LogFatal(strerror(errno));
}
if (-1 == ::listen(sock, LISTEN_QUEUE_LEN)) {
LogFatal(strerror(errno));
}
return sock;
}
const char* const HTTP_FORMAT = "HTTP/1.1 200 OK\r\nConnection: close\r\nServer: HuskyServer/1.0.0\r\nContent-Type: text/json; charset=%s\r\nContent-Length: %d\r\n\r\n%s";
const char* const CHARSET_UTF8 = "UTF-8";
} // namespace husky
#endif

View File

@ -1,41 +0,0 @@
#ifndef HUSKY_THREADPOOLSERVER_H
#define HUSKY_THREADPOOLSERVER_H
#include "NetUtils.hpp"
#include "WorkerThread.hpp"
namespace husky {
using namespace limonp;
class ThreadPoolServer {
public:
ThreadPoolServer(size_t thread_number, size_t queue_max_size, size_t port, IRequestHandler & handler):
_pool(thread_number, queue_max_size), _reqHandler(handler), _host_socket(-1) {
_host_socket = CreateAndListenSocket(port);
}
~ThreadPoolServer() {};
bool start() {
_pool.start();
sockaddr_in clientaddr;
socklen_t nSize = sizeof(clientaddr);
int clientSock;
while(true) {
if(-1 == (clientSock = accept(_host_socket, (struct sockaddr*) &clientaddr, &nSize))) {
LogError(strerror(errno));
break;
}
_pool.add(CreateTask<WorkerThread,int, IRequestHandler&>(clientSock, _reqHandler));
}
return true;
}
private:
ThreadPool _pool;
IRequestHandler & _reqHandler;
int _host_socket;
}; // class ThreadPoolServer
} // namespace husky
#endif

View File

@ -1,107 +0,0 @@
#ifndef HUSKY_WORKER_HPP
#define HUSKY_WORKER_HPP
#include "limonp/ThreadPool.hpp"
#include "IRequestHandler.hpp"
#include "NetUtils.hpp"
namespace husky {
const char* const CLIENT_IP_K = "CLIENT_IP";
const size_t RECV_BUFFER_SIZE = 16 * 1024;
const struct linger LNG = {1, 1};
const struct timeval SOCKET_TIMEOUT = {16, 0};
class WorkerThread: public ITask {
public:
WorkerThread(int sockfs, IRequestHandler& reqHandler):
_sockfd(sockfs), _reqHandler(reqHandler) {
}
virtual ~WorkerThread() {
}
public:
void run() {
do {
if(!_setsockopt(_sockfd)) {
LogError("_getsockopt failed.");
break;
}
string strSnd, strRetByHandler;
HttpReqInfo httpReq;
if(!_receive(_sockfd, httpReq)) {
LogError("_receive failed.");
break;
}
if(httpReq.isGET() && !_reqHandler.doGET(httpReq, strRetByHandler)) {
LogError("doGET failed.");
break;
}
if(httpReq.isPOST() && !_reqHandler.doPOST(httpReq, strRetByHandler)) {
LogError("doPOST failed.");
break;
}
strSnd = string_format(HTTP_FORMAT, CHARSET_UTF8, strRetByHandler.length(), strRetByHandler.c_str());
if(!_send(_sockfd, strSnd)) {
LogError("_send failed.");
break;
}
LogInfo("response:%s", strRetByHandler.c_str());
} while(false);
if(-1 == close(_sockfd)) {
LogError(strerror(errno));
}
}
private:
bool _receive(int sockfd, HttpReqInfo& httpInfo) const {
char recvBuf[RECV_BUFFER_SIZE];
int n = 0;
while(!httpInfo.isBodyFinished() && (n = recv(sockfd, recvBuf, RECV_BUFFER_SIZE, 0)) > 0) {
if(!httpInfo.isHeaderFinished()) {
if(!httpInfo.parseHeader(recvBuf, n)) {
LogError("parseHeader failed. ");
return false;
}
continue;
}
httpInfo.appendBody(recvBuf, n);
}
if(n < 0) {
LogError(strerror(errno));
return false;
}
return true;
}
bool _send(int sockfd, const string& strSnd) const {
if(-1 == send(sockfd, strSnd.c_str(), strSnd.length(), 0)) {
LogError(strerror(errno));
return false;
}
return true;
}
bool _setsockopt(int sockfd) const {
if(-1 == setsockopt(sockfd, SOL_SOCKET, SO_LINGER, (const char*)&LNG, sizeof(LNG))) {
LogError(strerror(errno));
return false;
}
if(-1 == setsockopt(sockfd, SOL_SOCKET, SO_RCVTIMEO, (const char*)&SOCKET_TIMEOUT, sizeof(SOCKET_TIMEOUT))) {
LogError(strerror(errno));
return false;
}
if(-1 == setsockopt(sockfd, SOL_SOCKET, SO_SNDTIMEO, (const char*)&SOCKET_TIMEOUT, sizeof(SOCKET_TIMEOUT))) {
LogError(strerror(errno));
return false;
}
return true;
}
private:
int _sockfd;
IRequestHandler& _reqHandler;
};
}
#endif

View File

@ -1,102 +0,0 @@
#include <unistd.h>
#include <algorithm>
#include <string>
#include <ctype.h>
#include <string.h>
#include "limonp/Config.hpp"
#include "husky/ThreadPoolServer.hpp"
#include "Jieba.hpp"
using namespace husky;
using namespace cppjieba;
class ReqHandler: public IRequestHandler {
public:
ReqHandler(const cppjieba::Jieba& jieba)
: jieba_(jieba) {
}
virtual ~ReqHandler() {
}
virtual bool doGET(const HttpReqInfo& httpReq, string& strSnd) {
string sentence, method, format;
string tmp;
vector<string> words;
httpReq.GET("key", tmp);
URLDecode(tmp, sentence);
httpReq.GET("method", method);
jieba_.Cut(sentence, words, true);
httpReq.GET("format", format);
Run(sentence, method, format, strSnd);
return true;
}
virtual bool doPOST(const HttpReqInfo& httpReq, string& strSnd) {
vector<string> words;
Run(httpReq.getBody(), "MIX", "simple", strSnd);
return true;
}
void Run(const string& sentence,
const string& method,
const string& format,
string& strSnd) const {
vector<string> words;
if ("MP" == method) {
jieba_.Cut(sentence, words, false);
} else if ("HMM" == method) {
jieba_.CutHMM(sentence, words);
} else if ("MIX" == method) {
jieba_.Cut(sentence, words, true);
} else if ("FULL" == method) {
jieba_.CutAll(sentence, words);
} else if ("QUERY" == method) {
jieba_.CutForSearch(sentence, words);
} else { // default
jieba_.Cut(sentence, words, false);
}
if (format == "simple") {
join(words.begin(), words.end(), strSnd, " ");
} else {
strSnd << words;
}
}
private:
const cppjieba::Jieba& jieba_;
};
bool Run(int argc, char** argv) {
if (argc < 2) {
return false;
}
Config conf(argv[1]);
if (!conf) {
return false;
}
int port = conf.get("port", 1339);
int threadNumber = conf.get("thread_number", 4);
int queueMaxSize = conf.get("queue_max_size", 1024);
string dictPath = conf.get("dict_path", "");
string modelPath = conf.get("model_path", "");
string userDictPath = conf.get("user_dict_path", "");
LogInfo("config info: %s", conf.getConfigInfo().c_str());
cppjieba::Jieba jieba(dictPath,
modelPath,
userDictPath);
ReqHandler reqHandler(jieba);
ThreadPoolServer sf(threadNumber, queueMaxSize, port, reqHandler);
return sf.start();
}
int main(int argc, char* argv[]) {
if (!Run(argc, argv)) {
printf("usage: %s <config_file>\n", argv[0]);
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}

View File

@ -1,218 +0,0 @@
#ifndef CPPJIEBA_DICT_TRIE_HPP
#define CPPJIEBA_DICT_TRIE_HPP
#include <iostream>
#include <fstream>
#include <map>
#include <cstring>
#include <stdint.h>
#include <cmath>
#include <limits>
#include "limonp/StringUtil.hpp"
#include "limonp/Logger.hpp"
#include "TransCode.hpp"
#include "Trie.hpp"
namespace cppjieba {
using namespace limonp;
const double MIN_DOUBLE = -3.14e+100;
const double MAX_DOUBLE = 3.14e+100;
const size_t DICT_COLUMN_NUM = 3;
const char* const UNKNOWN_TAG = "";
class DictTrie {
public:
enum UserWordWeightOption {
WordWeightMin,
WordWeightMedian,
WordWeightMax,
}; // enum UserWordWeightOption
DictTrie(const string& dict_path, const string& user_dict_paths = "", UserWordWeightOption user_word_weight_opt = WordWeightMedian) {
Init(dict_path, user_dict_paths, user_word_weight_opt);
}
~DictTrie() {
delete trie_;
}
bool InsertUserWord(const string& word, const string& tag = UNKNOWN_TAG) {
DictUnit node_info;
if (!MakeNodeInfo(node_info, word, user_word_default_weight_, tag)) {
return false;
}
active_node_infos_.push_back(node_info);
trie_->InsertNode(node_info.word, &active_node_infos_.back());
return true;
}
const DictUnit* Find(Unicode::const_iterator begin, Unicode::const_iterator end) const {
return trie_->Find(begin, end);
}
void Find(Unicode::const_iterator begin,
Unicode::const_iterator end,
vector<struct Dag>&res,
size_t max_word_len = MAX_WORD_LENGTH) const {
trie_->Find(begin, end, res, max_word_len);
}
bool IsUserDictSingleChineseWord(const Rune& word) const {
return isIn(user_dict_single_chinese_word_, word);
}
double GetMinWeight() const {
return min_weight_;
}
private:
void Init(const string& dict_path, const string& user_dict_paths, UserWordWeightOption user_word_weight_opt) {
LoadDict(dict_path);
CalculateWeight(static_node_infos_);
SetStaticWordWeights(user_word_weight_opt);
if (user_dict_paths.size()) {
LoadUserDict(user_dict_paths);
}
Shrink(static_node_infos_);
CreateTrie(static_node_infos_);
}
void CreateTrie(const vector<DictUnit>& dictUnits) {
assert(dictUnits.size());
vector<Unicode> words;
vector<const DictUnit*> valuePointers;
for (size_t i = 0 ; i < dictUnits.size(); i ++) {
words.push_back(dictUnits[i].word);
valuePointers.push_back(&dictUnits[i]);
}
trie_ = new Trie(words, valuePointers);
}
void LoadUserDict(const string& filePaths) {
vector<string> files = limonp::split(filePaths, "|;");
size_t lineno = 0;
for (size_t i = 0; i < files.size(); i++) {
ifstream ifs(files[i].c_str());
if (!ifs.is_open()) {
LogFatal("file %s open failed.", files[i].c_str());
}
string line;
DictUnit node_info;
vector<string> buf;
for (; getline(ifs, line); lineno++) {
if (line.size() == 0)
continue;
buf.clear();
split(line, buf, " ");
if (buf.size() < 1) {
LogFatal("split [%s] result illegal", line.c_str());
}
DictUnit node_info;
MakeNodeInfo(node_info,
buf[0],
user_word_default_weight_,
(buf.size() == 2 ? buf[1] : UNKNOWN_TAG));
static_node_infos_.push_back(node_info);
if (node_info.word.size() == 1) {
user_dict_single_chinese_word_.insert(node_info.word[0]);
}
}
}
LogInfo("Load userdicts[%s] ok. lines[%u]", filePaths.c_str(), lineno);
}
bool MakeNodeInfo(DictUnit& node_info,
const string& word,
double weight,
const string& tag) {
if (!TransCode::Decode(word, node_info.word)) {
LogError("Decode %s failed.", word.c_str());
return false;
}
node_info.weight = weight;
node_info.tag = tag;
return true;
}
void LoadDict(const string& filePath) {
ifstream ifs(filePath.c_str());
if (!ifs.is_open()) {
LogFatal("file %s open failed.", filePath.c_str());
}
string line;
vector<string> buf;
DictUnit node_info;
for (size_t lineno = 0; getline(ifs, line); lineno++) {
split(line, buf, " ");
if (buf.size() != DICT_COLUMN_NUM) {
LogFatal("split result illegal, line: %s, result size: %u", line.c_str(), buf.size());
}
MakeNodeInfo(node_info,
buf[0],
atof(buf[1].c_str()),
buf[2]);
static_node_infos_.push_back(node_info);
}
}
static bool WeightCompare(const DictUnit& lhs, const DictUnit& rhs) {
return lhs.weight < rhs.weight;
}
void SetStaticWordWeights(UserWordWeightOption option) {
if (static_node_infos_.empty()) {
LogFatal("something must be wrong");
}
vector<DictUnit> x = static_node_infos_;
sort(x.begin(), x.end(), WeightCompare);
min_weight_ = x[0].weight;
max_weight_ = x[x.size() - 1].weight;
median_weight_ = x[x.size() / 2].weight;
switch (option) {
case WordWeightMin:
user_word_default_weight_ = min_weight_;
break;
case WordWeightMedian:
user_word_default_weight_ = median_weight_;
break;
default:
user_word_default_weight_ = max_weight_;
break;
}
}
void CalculateWeight(vector<DictUnit>& node_infos) const {
double sum = 0.0;
for (size_t i = 0; i < node_infos.size(); i++) {
sum += node_infos[i].weight;
}
assert(sum);
for (size_t i = 0; i < node_infos.size(); i++) {
DictUnit& node_info = node_infos[i];
assert(node_info.weight);
node_info.weight = log(double(node_info.weight)/double(sum));
}
}
void Shrink(vector<DictUnit>& units) const {
vector<DictUnit>(units.begin(), units.end()).swap(units);
}
vector<DictUnit> static_node_infos_;
deque<DictUnit> active_node_infos_; // must not be vector
Trie * trie_;
double min_weight_;
double max_weight_;
double median_weight_;
double user_word_default_weight_;
unordered_set<Rune> user_dict_single_chinese_word_;
};
}
#endif

View File

@ -1,100 +0,0 @@
#ifndef CPPJIEAB_JIEBA_H
#define CPPJIEAB_JIEBA_H
#include "QuerySegment.hpp"
#include "PosTagger.hpp"
#include "LevelSegment.hpp"
namespace cppjieba {
class Jieba {
public:
Jieba(const string& dict_path, const string& model_path, const string& user_dict_path)
: dict_trie_(dict_path, user_dict_path),
model_(model_path),
mp_seg_(&dict_trie_),
hmm_seg_(&model_),
mix_seg_(&dict_trie_, &model_),
full_seg_(&dict_trie_),
query_seg_(&dict_trie_, &model_),
level_seg_(&dict_trie_),
pos_tagger_(&dict_trie_, &model_) {
}
~Jieba() {
}
struct LocWord {
string word;
size_t begin;
size_t end;
}; // struct LocWord
void Cut(const string& sentence, vector<string>& words, bool hmm = true) const {
mix_seg_.Cut(sentence, words, hmm);
}
void CutAll(const string& sentence, vector<string>& words) const {
full_seg_.Cut(sentence, words);
}
void CutForSearch(const string& sentence, vector<string>& words, bool hmm = true) const {
query_seg_.Cut(sentence, words, hmm);
}
void CutHMM(const string& sentence, vector<string>& words) const {
hmm_seg_.Cut(sentence, words);
}
void CutLevel(const string& sentence, vector<string>& words) const {
level_seg_.Cut(sentence, words);
}
void CutLevel(const string& sentence, vector<pair<string, size_t> >& words) const {
level_seg_.Cut(sentence, words);
}
void CutSmall(const string& sentence, vector<string>& words, size_t max_word_len) const {
mp_seg_.Cut(sentence, words, max_word_len);
}
void Locate(const vector<string>& words, vector<LocWord>& loc_words) const {
loc_words.resize(words.size());
size_t begin = 0;
for (size_t i = 0; i < words.size(); i++) {
size_t len = TransCode::Decode(words[i]).size();
loc_words[i].word = words[i];
loc_words[i].begin = begin;
loc_words[i].end = loc_words[i].begin + len;
begin = loc_words[i].end;
}
}
void Tag(const string& sentence, vector<pair<string, string> >& words) const {
pos_tagger_.Tag(sentence, words);
}
bool InsertUserWord(const string& word, const string& tag = UNKNOWN_TAG) {
return dict_trie_.InsertUserWord(word, tag);
}
const DictTrie* GetDictTrie() const {
return &dict_trie_;
}
const HMMModel* GetHMMModel() const {
return &model_;
}
void SetQuerySegmentThreshold(size_t len) {
query_seg_.SetMaxWordLen(len);
}
private:
DictTrie dict_trie_;
HMMModel model_;
// They share the same dict trie and model
MPSegment mp_seg_;
HMMSegment hmm_seg_;
MixSegment mix_seg_;
FullSegment full_seg_;
QuerySegment query_seg_;
LevelSegment level_seg_;
PosTagger pos_tagger_;
}; // class Jieba
} // namespace Jieba
#endif // CPPJIEAB_JIEBA_H

View File

@ -1,145 +0,0 @@
#ifndef CPPJIEBA_KEYWORD_EXTRACTOR_H
#define CPPJIEBA_KEYWORD_EXTRACTOR_H
#include "MixSegment.hpp"
#include <cmath>
#include <set>
namespace cppjieba {
using namespace limonp;
/*utf8*/
class KeywordExtractor {
public:
KeywordExtractor(const string& dictPath,
const string& hmmFilePath,
const string& idfPath,
const string& stopWordPath,
const string& userDict = "")
: segment_(dictPath, hmmFilePath, userDict) {
LoadIdfDict(idfPath);
LoadStopWordDict(stopWordPath);
}
KeywordExtractor(const DictTrie* dictTrie,
const HMMModel* model,
const string& idfPath,
const string& stopWordPath)
: segment_(dictTrie, model){
LoadIdfDict(idfPath);
LoadStopWordDict(stopWordPath);
}
~KeywordExtractor() {
}
bool Extract(const string& sentence, vector<string>& keywords, size_t topN) const {
vector<pair<string, double> > topWords;
if (!Extract(sentence, topWords, topN)) {
return false;
}
for (size_t i = 0; i < topWords.size(); i++) {
keywords.push_back(topWords[i].first);
}
return true;
}
bool Extract(const string& sentence, vector<pair<string, double> >& keywords, size_t topN) const {
vector<string> words;
segment_.Cut(sentence, words);
map<string, double> wordmap;
for (vector<string>::iterator iter = words.begin(); iter != words.end(); iter++) {
if (IsSingleWord(*iter)) {
continue;
}
wordmap[*iter] += 1.0;
}
for (map<string, double>::iterator itr = wordmap.begin(); itr != wordmap.end(); ) {
if (stopWords_.end() != stopWords_.find(itr->first)) {
wordmap.erase(itr++);
continue;
}
unordered_map<string, double>::const_iterator cit = idfMap_.find(itr->first);
if (cit != idfMap_.end()) {
itr->second *= cit->second;
} else {
itr->second *= idfAverage_;
}
itr ++;
}
keywords.clear();
std::copy(wordmap.begin(), wordmap.end(), std::inserter(keywords, keywords.begin()));
topN = min(topN, keywords.size());
partial_sort(keywords.begin(), keywords.begin() + topN, keywords.end(), Compare);
keywords.resize(topN);
return true;
}
private:
void LoadIdfDict(const string& idfPath) {
ifstream ifs(idfPath.c_str());
if (!ifs.is_open()) {
LogFatal("open %s failed.", idfPath.c_str());
}
string line ;
vector<string> buf;
double idf = 0.0;
double idfSum = 0.0;
size_t lineno = 0;
for (; getline(ifs, line); lineno++) {
buf.clear();
if (line.empty()) {
LogError("line[%d] empty. skipped.", lineno);
continue;
}
split(line, buf, " ");
if (buf.size() != 2) {
LogError("line %d [%s] illegal. skipped.", lineno, line.c_str());
continue;
}
idf = atof(buf[1].c_str());
idfMap_[buf[0]] = idf;
idfSum += idf;
}
assert(lineno);
idfAverage_ = idfSum / lineno;
assert(idfAverage_ > 0.0);
}
void LoadStopWordDict(const string& filePath) {
ifstream ifs(filePath.c_str());
if (!ifs.is_open()) {
LogFatal("open %s failed.", filePath.c_str());
}
string line ;
while (getline(ifs, line)) {
stopWords_.insert(line);
}
assert(stopWords_.size());
}
bool IsSingleWord(const string& str) const {
Unicode unicode;
TransCode::Decode(str, unicode);
if (unicode.size() == 1)
return true;
return false;
}
static bool Compare(const pair<string, double>& lhs, const pair<string, double>& rhs) {
return lhs.second > rhs.second;
}
MixSegment segment_;
unordered_map<string, double> idfMap_;
double idfAverage_;
unordered_set<string> stopWords_;
};
}
#endif

View File

@ -1,82 +0,0 @@
#ifndef CPPJIEBA_LEVELSEGMENT_H
#define CPPJIEBA_LEVELSEGMENT_H
#include "MPSegment.hpp"
namespace cppjieba {
class LevelSegment: public SegmentBase{
public:
LevelSegment(const string& dictPath,
const string& userDictPath = "")
: mpSeg_(dictPath, userDictPath) {
LogInfo("LevelSegment init");
}
LevelSegment(const DictTrie* dictTrie)
: mpSeg_(dictTrie) {
}
~LevelSegment() {
}
void Cut(Unicode::const_iterator begin,
Unicode::const_iterator end,
vector<pair<Unicode, size_t> >& res) const {
res.clear();
vector<Unicode> words;
vector<Unicode> smallerWords;
words.reserve(end - begin);
mpSeg_.Cut(begin, end, words);
smallerWords.reserve(words.size());
res.reserve(words.size());
size_t level = 0;
while (!words.empty()) {
smallerWords.clear();
for (size_t i = 0; i < words.size(); i++) {
if (words[i].size() >= 3) {
size_t len = words[i].size() - 1;
mpSeg_.Cut(words[i].begin(), words[i].end(), smallerWords, len); // buffer.push_back without clear
}
if (words[i].size() > 1) {
res.push_back(pair<Unicode, size_t>(words[i], level));
}
}
words.swap(smallerWords);
level++;
}
}
void Cut(const string& sentence,
vector<pair<string, size_t> >& words) const {
words.clear();
Unicode unicode;
TransCode::Decode(sentence, unicode);
vector<pair<Unicode, size_t> > unicodeWords;
Cut(unicode.begin(), unicode.end(), unicodeWords);
words.resize(unicodeWords.size());
for (size_t i = 0; i < words.size(); i++) {
TransCode::Encode(unicodeWords[i].first, words[i].first);
words[i].second = unicodeWords[i].second;
}
}
bool Cut(const string& sentence,
vector<string>& res) const {
vector<pair<string, size_t> > words;
Cut(sentence, words);
res.clear();
res.reserve(words.size());
for (size_t i = 0; i < words.size(); i++) {
res.push_back(words[i].first);
}
return true;
}
private:
MPSegment mpSeg_;
}; // class LevelSegment
} // namespace cppjieba
#endif // CPPJIEBA_LEVELSEGMENT_H

View File

@ -1,83 +0,0 @@
#ifndef CPPJIEBA_QUERYSEGMENT_H
#define CPPJIEBA_QUERYSEGMENT_H
#include <algorithm>
#include <set>
#include <cassert>
#include "limonp/Logger.hpp"
#include "DictTrie.hpp"
#include "SegmentBase.hpp"
#include "FullSegment.hpp"
#include "MixSegment.hpp"
#include "TransCode.hpp"
#include "DictTrie.hpp"
namespace cppjieba {
class QuerySegment: public SegmentBase {
public:
QuerySegment(const string& dict, const string& model, const string& userDict = "", size_t maxWordLen = 4)
: mixSeg_(dict, model, userDict),
fullSeg_(mixSeg_.GetDictTrie()),
maxWordLen_(maxWordLen) {
assert(maxWordLen_);
}
QuerySegment(const DictTrie* dictTrie, const HMMModel* model, size_t maxWordLen = 4)
: mixSeg_(dictTrie, model), fullSeg_(dictTrie), maxWordLen_(maxWordLen) {
}
~QuerySegment() {
}
void Cut(const string& sentence, vector<string>& words, bool hmm = true) const {
PreFilter pre_filter(symbols_, sentence);
PreFilter::Range range;
vector<Unicode> uwords;
uwords.reserve(sentence.size());
while (pre_filter.HasNext()) {
range = pre_filter.Next();
Cut(range.begin, range.end, uwords, hmm);
}
TransCode::Encode(uwords, words);
}
void Cut(Unicode::const_iterator begin, Unicode::const_iterator end, vector<Unicode>& res, bool hmm) const {
//use mix Cut first
vector<Unicode> mixRes;
mixSeg_.Cut(begin, end, mixRes, hmm);
vector<Unicode> fullRes;
for (vector<Unicode>::const_iterator mixResItr = mixRes.begin(); mixResItr != mixRes.end(); mixResItr++) {
// if it's too long, Cut with fullSeg_, put fullRes in res
if (mixResItr->size() > maxWordLen_ && !IsAllAscii(*mixResItr)) {
fullSeg_.Cut(mixResItr->begin(), mixResItr->end(), fullRes);
for (vector<Unicode>::const_iterator fullResItr = fullRes.begin(); fullResItr != fullRes.end(); fullResItr++) {
res.push_back(*fullResItr);
}
//clear tmp res
fullRes.clear();
} else { // just use the mix result
res.push_back(*mixResItr);
}
}
}
void SetMaxWordLen(size_t len) {
maxWordLen_ = len;
}
size_t GetMaxWordLen() const {
return maxWordLen_;
}
private:
bool IsAllAscii(const Unicode& s) const {
for(size_t i = 0; i < s.size(); i++) {
if (s[i] >= 0x80) {
return false;
}
}
return true;
}
MixSegment mixSeg_;
FullSegment fullSeg_;
size_t maxWordLen_;
}; // QuerySegment
} // namespace cppjieba
#endif

View File

@ -1,37 +0,0 @@
#ifndef CPPJIEBA_SEGMENTBASE_H
#define CPPJIEBA_SEGMENTBASE_H
#include "limonp/Logger.hpp"
#include "PreFilter.hpp"
#include <cassert>
namespace cppjieba {
//const char* const SPECIAL_CHARS = " \t\n。";
const Rune SPECIAL_SYMBOL[] = {32u, 9u, 10u, 65292u, 12290u};
using namespace limonp;
class SegmentBase {
public:
SegmentBase() {
LoadSpecialSymbols();
}
~SegmentBase() {
}
protected:
void LoadSpecialSymbols() {
size_t size = sizeof(SPECIAL_SYMBOL)/sizeof(*SPECIAL_SYMBOL);
for (size_t i = 0; i < size; i ++) {
symbols_.insert(SPECIAL_SYMBOL[i]);
}
assert(symbols_.size());
}
unordered_set<Rune> symbols_;
}; // class SegmentBase
} // cppjieba
#endif

View File

@ -1,70 +0,0 @@
/************************************
* file enc : utf-8
* author : wuyanyi09@gmail.com
************************************/
#ifndef CPPJIEBA_TRANSCODE_H
#define CPPJIEBA_TRANSCODE_H
#include "limonp/StringUtil.hpp"
#include "limonp/LocalVector.hpp"
namespace cppjieba {
using namespace limonp;
typedef uint16_t Rune;
typedef limonp::LocalVector<Rune> Unicode;
namespace TransCode {
inline bool Decode(const string& str, Unicode& res) {
#ifdef CPPJIEBA_GBK
return gbkTrans(str, res);
#else
return utf8ToUnicode(str, res);
#endif
}
inline void Encode(Unicode::const_iterator begin, Unicode::const_iterator end, string& res) {
#ifdef CPPJIEBA_GBK
gbkTrans(begin, end, res);
#else
unicodeToUtf8(begin, end, res);
#endif
}
inline void Encode(const Unicode& uni, string& res) {
Encode(uni.begin(), uni.end(), res);
}
// compiler is expected to optimized this function to avoid return value copy
inline string Encode(Unicode::const_iterator begin, Unicode::const_iterator end) {
string res;
res.reserve(end - begin);
Encode(begin, end, res);
return res;
}
inline string Encode(const Unicode& unicode) {
return Encode(unicode.begin(), unicode.end());
}
// compiler is expected to optimized this function to avoid return value copy
inline Unicode Decode(const string& str) {
Unicode unicode;
unicode.reserve(str.size());
Decode(str, unicode);
return unicode;
}
inline void Encode(const vector<Unicode>& input, vector<string>& output) {
output.resize(input.size());
for (size_t i = 0; i < output.size(); i++) {
Encode(input[i], output[i]);
}
}
} // namespace TransCode
} // namespace cppjieba
#endif

View File

@ -1,176 +0,0 @@
#ifndef CPPJIEBA_TRIE_HPP
#define CPPJIEBA_TRIE_HPP
#include "limonp/StdExtension.hpp"
#include <vector>
#include <queue>
namespace cppjieba {
using namespace std;
const size_t MAX_WORD_LENGTH = 512;
struct DictUnit {
Unicode word;
double weight;
string tag;
};
// for debugging
inline ostream & operator << (ostream& os, const DictUnit& unit) {
string s;
s << unit.word;
return os << string_format("%s %s %.3lf", s.c_str(), unit.tag.c_str(), unit.weight);
}
struct Dag {
uint16_t rune;
LocalVector<pair<size_t, const DictUnit*> > nexts;
const DictUnit * pInfo;
double weight;
size_t nextPos;
Dag():rune(0), pInfo(NULL), weight(0.0), nextPos(0) {
}
};
typedef Rune TrieKey;
class TrieNode {
public :
TrieNode(): next(NULL), ptValue(NULL) {
}
public:
typedef unordered_map<TrieKey, TrieNode*> NextMap;
NextMap *next;
const DictUnit *ptValue;
};
class Trie {
public:
static const size_t BASE_SIZE = (1 << (8 * (sizeof(TrieKey))));
Trie(const vector<Unicode>& keys, const vector<const DictUnit*>& valuePointers) {
CreateTrie(keys, valuePointers);
}
~Trie() {
for (size_t i = 0; i < BASE_SIZE; i++) {
if (_base[i].next == NULL) {
continue;
}
for (TrieNode::NextMap::iterator it = _base[i].next->begin(); it != _base[i].next->end(); it++) {
DeleteNode(it->second);
it->second = NULL;
}
delete _base[i].next;
_base[i].next = NULL;
}
}
const DictUnit* Find(Unicode::const_iterator begin, Unicode::const_iterator end) const {
if (begin == end) {
return NULL;
}
const TrieNode* ptNode = _base + (*(begin++));
TrieNode::NextMap::const_iterator citer;
for (Unicode::const_iterator it = begin; it != end; it++) {
if (NULL == ptNode->next) {
return NULL;
}
citer = ptNode->next->find(*it);
if (ptNode->next->end() == citer) {
return NULL;
}
ptNode = citer->second;
}
return ptNode->ptValue;
}
void Find(Unicode::const_iterator begin,
Unicode::const_iterator end,
vector<struct Dag>&res,
size_t max_word_len = MAX_WORD_LENGTH) const {
res.resize(end - begin);
const TrieNode *ptNode = NULL;
TrieNode::NextMap::const_iterator citer;
for (size_t i = 0; i < size_t(end - begin); i++) {
Rune rune = *(begin + i);
ptNode = _base + rune;
res[i].rune = rune;
assert(res[i].nexts.empty());
res[i].nexts.push_back(pair<size_t, const DictUnit*>(i, ptNode->ptValue));
for (size_t j = i + 1; j < size_t(end - begin) && (j - i + 1) <= max_word_len ; j++) {
if (ptNode->next == NULL) {
break;
}
citer = ptNode->next->find(*(begin + j));
if (ptNode->next->end() == citer) {
break;
}
ptNode = citer->second;
if (NULL != ptNode->ptValue) {
res[i].nexts.push_back(pair<size_t, const DictUnit*>(j, ptNode->ptValue));
}
}
}
}
void InsertNode(const Unicode& key, const DictUnit* ptValue) {
if (key.begin() == key.end()) {
return;
}
TrieNode::NextMap::const_iterator kmIter;
Unicode::const_iterator citer= key.begin();
TrieNode *ptNode = _base + (*(citer++));
for (; citer != key.end(); citer++) {
if (NULL == ptNode->next) {
ptNode->next = new TrieNode::NextMap;
}
kmIter = ptNode->next->find(*citer);
if (ptNode->next->end() == kmIter) {
TrieNode *nextNode = new TrieNode;
(*(ptNode->next))[*citer] = nextNode;
ptNode = nextNode;
} else {
ptNode = kmIter->second;
}
}
ptNode->ptValue = ptValue;
}
private:
void CreateTrie(const vector<Unicode>& keys, const vector<const DictUnit*>& valuePointers) {
if (valuePointers.empty() || keys.empty()) {
return;
}
assert(keys.size() == valuePointers.size());
for (size_t i = 0; i < keys.size(); i++) {
InsertNode(keys[i], valuePointers[i]);
}
}
void DeleteNode(TrieNode* node) {
if (NULL == node) {
return;
}
if (NULL != node->next) {
TrieNode::NextMap::iterator it;
for (it = node->next->begin(); it != node->next->end(); it++) {
DeleteNode(it->second);
}
delete node->next;
node->next = NULL;
}
delete node;
}
TrieNode _base[BASE_SIZE];
};
}
#endif

View File

@ -1,67 +0,0 @@
/************************************
* file enc : ascii
* author : wuyanyi09@gmail.com
************************************/
#ifndef LIMONP_ARGV_FUNCTS_H
#define LIMONP_ARGV_FUNCTS_H
#include <set>
#include <sstream>
#include "StringUtil.hpp"
namespace limonp {
using namespace std;
class ArgvContext {
public :
ArgvContext(int argc, const char* const * argv) {
for(int i = 0; i < argc; i++) {
if(startsWith(argv[i], "-")) {
if(i + 1 < argc && !startsWith(argv[i + 1], "-")) {
mpss_[argv[i]] = argv[i+1];
i++;
} else {
sset_.insert(argv[i]);
}
} else {
args_.push_back(argv[i]);
}
}
}
~ArgvContext() {};
public:
friend ostream& operator << (ostream& os, const ArgvContext& args);
string operator [](size_t i) const {
if(i < args_.size()) {
return args_[i];
}
return "";
}
string operator [](const string& key) const {
map<string, string>::const_iterator it = mpss_.find(key);
if(it != mpss_.end()) {
return it->second;
}
return "";
}
public:
bool hasKey(const string& key) const {
if(mpss_.find(key) != mpss_.end() || sset_.find(key) != sset_.end()) {
return true;
}
return false;
}
private:
vector<string> args_;
map<string, string> mpss_;
set<string> sset_;
};
inline ostream& operator << (ostream& os, const ArgvContext& args) {
return os<<args.args_<<args.mpss_<<args.sset_;
}
}
#endif

View File

@ -1,111 +0,0 @@
/*
https://github.com/chenshuo/muduo/blob/master/muduo/base/BlockingQueue.h
*/
#ifndef LIMONP_BLOCKINGQUEUE_HPP
#define LIMONP_BLOCKINGQUEUE_HPP
#include <queue>
#include "BoundedQueue.hpp"
#include "Condition.hpp"
namespace limonp {
template<class T>
class BlockingQueue: NonCopyable {
public:
BlockingQueue()
: mutex_(), notEmpty_(mutex_), queue_() {
}
void push(const T& x) {
MutexLockGuard lock(mutex_);
queue_.push(x);
notEmpty_.notify(); // wait morphing saves us
}
T pop() {
MutexLockGuard lock(mutex_);
// always use a while-loop, due to spurious wakeup
while (queue_.empty()) {
notEmpty_.wait();
}
assert(!queue_.empty());
T front(queue_.front());
queue_.pop();
return front;
}
size_t size() const {
MutexLockGuard lock(mutex_);
return queue_.size();
}
bool empty() const {
return size() == 0;
}
private:
mutable MutexLock mutex_;
Condition notEmpty_;
std::queue<T> queue_;
};
template<typename T>
class BoundedBlockingQueue : NonCopyable {
public:
explicit BoundedBlockingQueue(size_t maxSize)
: mutex_(),
notEmpty_(mutex_),
notFull_(mutex_),
queue_(maxSize) {
}
void push(const T& x) {
MutexLockGuard lock(mutex_);
while (queue_.full()) {
notFull_.wait();
}
assert(!queue_.full());
queue_.push(x);
notEmpty_.notify();
}
T pop() {
MutexLockGuard lock(mutex_);
while (queue_.empty()) {
notEmpty_.wait();
}
assert(!queue_.empty());
T res = queue_.pop();
notFull_.notify();
return res;
}
bool empty() const {
MutexLockGuard lock(mutex_);
return queue_.empty();
}
bool full() const {
MutexLockGuard lock(mutex_);
return queue_.full();
}
size_t size() const {
MutexLockGuard lock(mutex_);
return queue_.size();
}
size_t capacity() const {
return queue_.capacity();
}
private:
mutable MutexLock mutex_;
Condition notEmpty_;
Condition notFull_;
BoundedQueue<T> queue_;
};
}
#endif

View File

@ -1,63 +0,0 @@
#ifndef LIMONP_BOUNDED_QUEUE_HPP
#define LIMONP_BOUNDED_QUEUE_HPP
#include <vector>
#include <fstream>
#include <cassert>
namespace limonp {
using namespace std;
template<class T>
class BoundedQueue {
private:
size_t head_;
size_t tail_;
size_t size_;
const size_t capacity_;
vector<T> circular__buffer;
public:
explicit BoundedQueue(size_t capacity): capacity_(capacity), circular__buffer(capacity) {
head_ = 0;
tail_ = 0;
size_ = 0;
assert(capacity_);
}
~BoundedQueue() {}
public:
void clear() {
head_ = 0;
tail_ = 0;
size_ = 0;
}
bool empty() const {
return !size_;
}
bool full() const {
return capacity_ == size_;
}
size_t size() const {
return size_;
}
size_t capacity() const {
return capacity_;
}
void push(const T& t) {
assert(!full());
circular__buffer[tail_] = t;
tail_ = (tail_ + 1) % capacity_;
size_ ++;
}
T pop() {
assert(!empty());
size_t oldPos = head_;
head_ = (head_ + 1) % capacity_;
size_ --;
return circular__buffer[oldPos];
}
};
}
#endif

View File

@ -1,82 +0,0 @@
#ifndef LIMONP_CAST_FUNCTS_H
#define LIMONP_CAST_FUNCTS_H
namespace limonp {
namespace CastFloat {
//logical and or
static const int sign_32 = 0xC0000000;
static const int exponent_32 = 0x07800000;
static const int mantissa_32 = 0x007FE000;
static const int sign_exponent_32 = 0x40000000;
static const int loss_32 = 0x38000000;
static const short sign_16 = (short)0xC000;
static const short exponent_16 = (short)0x3C00;
static const short mantissa_16 = (short)0x03FF;
static const short sign_exponent_16 = (short)0x4000;
static const int exponent_fill_32 = 0x38000000;
//infinite
static const short infinite_16 = (short) 0x7FFF;
static const short infinitesmall_16 = (short) 0x0000;
inline float intBitsToFloat(unsigned int x) {
union {
float f;
int i;
} u;
u.i = x;
return u.f;
}
inline int floatToIntBits(float f) {
union {
float f;
int i ;
} u;
u.f = f;
return u.i;
}
inline short floatToShortBits(float f) {
int fi = floatToIntBits(f);
// 提取关键信息
short sign = (short) ((unsigned int)(fi & sign_32) >> 16);
short exponent = (short) ((unsigned int)(fi & exponent_32) >> 13);
short mantissa = (short) ((unsigned int)(fi & mantissa_32) >> 13);
// 生成编码结果
short code = (short) (sign | exponent | mantissa);
// 无穷大量、无穷小量的处理
if ((fi & loss_32) > 0 && (fi & sign_exponent_32) > 0) {
// 当指数符号为1时(正次方)且左234位为1返回无穷大量
return (short) (code | infinite_16);
}
if (((fi & loss_32) ^ loss_32) > 0 && (fi & sign_exponent_32) == 0) {
// 当指数符号位0时(负次方)且左234位为0(与111异或>0),返回无穷小量
return infinitesmall_16;
}
return code;
}
inline float shortBitsToFloat(short s) {
/*
* 31001 0(13)
*/
int sign = ((int) (s & sign_16)) << 16;
int exponent = ((int) (s & exponent_16)) << 13;
// 指数符号位为0234位补1
if ((s & sign_exponent_16) == 0 && s != 0) {
exponent |= exponent_fill_32;
}
int mantissa = ((int) (s & mantissa_16)) << 13;
// 生成解码结果
int code = sign | exponent | mantissa;
return intBitsToFloat(code);
}
}
}
#endif

View File

@ -1,36 +0,0 @@
#ifndef LIMONP_COLOR_PRINT_HPP
#define LIMONP_COLOR_PRINT_HPP
#include <string>
#include <stdarg.h>
namespace limonp {
using std::string;
enum COLOR {
BLACK = 30,
RED,
GREEN,
YELLOW,
BLUE,
PURPLE
};
static void ColorPrintln(enum COLOR color, const char * fmt, ...) {
va_list ap;
printf("\033[0;%dm", color);
va_start(ap, fmt);
vprintf(fmt, ap);
va_end(ap);
printf("\033[0m\n"); // if not \n , in some situation , the next lines will be set the same color unexpectedly
}
#if 0
static void ColorPrint(const string& str, enum COLOR color = GREEN) {
printf("\033[0;%dm%s\033[0m", color, str.c_str());
}
#endif
} // namespace limonp
#endif

View File

@ -1,41 +0,0 @@
/*
* https://github.com/chenshuo/muduo/blob/master/muduo/base/Condition.h
*/
#ifndef LIMONP_CONDITION_HPP
#define LIMONP_CONDITION_HPP
#include "MutexLock.hpp"
namespace limonp {
class Condition : NonCopyable {
public:
explicit Condition(MutexLock& mutex)
: mutex_(mutex) {
LIMONP_CHECK(!pthread_cond_init(&pcond_, NULL));
}
~Condition() {
LIMONP_CHECK(!pthread_cond_destroy(&pcond_));
}
void wait() {
LIMONP_CHECK(!pthread_cond_wait(&pcond_, mutex_.getPthreadMutex()));
}
void notify() {
LIMONP_CHECK(!pthread_cond_signal(&pcond_));
}
void notifyAll() {
LIMONP_CHECK(!pthread_cond_broadcast(&pcond_));
}
private:
MutexLock& mutex_;
pthread_cond_t pcond_;
};
}
#endif

View File

@ -1,100 +0,0 @@
/************************************
* file enc : utf8
* author : wuyanyi09@gmail.com
************************************/
#ifndef LIMONP_CONFIG_H
#define LIMONP_CONFIG_H
#include <map>
#include <fstream>
#include <iostream>
#include <assert.h>
#include "StringUtil.hpp"
namespace limonp {
using namespace std;
class Config {
public:
explicit Config(const string& filePath) {
loadFile_(filePath);
}
public:
operator bool () {
return !map_.empty();
}
private:
void loadFile_(const string& filePath) {
ifstream ifs(filePath.c_str());
assert(ifs);
string line;
vector<string> vecBuf;
size_t lineno = 0;
while(getline(ifs, line)) {
lineno ++;
trim(line);
if(line.empty() || startsWith(line, "#")) {
continue;
}
vecBuf.clear();
split(line, vecBuf, "=");
if(2 != vecBuf.size()) {
fprintf(stderr, "line[%s] illegal.\n", line.c_str());
assert(false);
continue;
}
string& key = vecBuf[0];
string& value = vecBuf[1];
trim(key);
trim(value);
if(!map_.insert(make_pair(key, value)).second) {
fprintf(stderr, "key[%s] already exits.\n", key.c_str());
assert(false);
continue;
}
}
ifs.close();
}
public:
string get(const string& key, const string& defaultvalue) const {
map<string, string>::const_iterator it = map_.find(key);
if(map_.end() != it) {
return it->second;
}
return defaultvalue;
}
int get(const string& key, int defaultvalue) const {
string str = get(key, "");
if("" == str) {
return defaultvalue;
}
return atoi(str.c_str());
}
const char* operator [] (const char* key) const {
if(NULL == key) {
return NULL;
}
map<string, string>::const_iterator it = map_.find(key);
if(map_.end() != it) {
return it->second.c_str();
}
return NULL;
}
public:
string getConfigInfo() const {
string res;
res << *this;
return res;
}
private:
map<string, string> map_;
private:
friend ostream& operator << (ostream& os, const Config& config);
};
inline ostream& operator << (ostream& os, const Config& config) {
return os << config.map_;
}
}
#endif

View File

@ -1,31 +0,0 @@
#ifndef LIMONP_HANDY_MACRO_HPP
#define LIMONP_HANDY_MACRO_HPP
#include <cstdio>
#include <cstdlib>
#define LIMONP_CHECK(exp) \
if(!(exp)){fprintf(stderr, "File:%s, Line:%d Exp:[" #exp "] is true, abort.\n", __FILE__, __LINE__); abort();}
#define print(x) cout << x << endl
/*
#define XX_GET_SET(varType, varName, funName)\
private: varType varName;\
public: inline varType get##funName(void) const {return varName;}\
public: inline void set##funName(varType var) {varName = var;}
#define XX_GET(varType, varName, funName)\
private: varType varName;\
public: inline varType get##funName(void) const {return varName;}
#define XX_SET(varType, varName, funName)\
private: varType varName;\
public: inline void set##funName(varType var) {varName = var;}
#define XX_GET_SET_BY_REF(varType, varName, funName)\
private: varType varName;\
public: inline const varType& get##funName(void) const {return varName;}\
public: inline void set##funName(const varType& var){varName = var;}
*/
#endif

View File

@ -1,25 +0,0 @@
#ifndef LIMONP_INITONOFF_H
#define LIMONP_INITONOFF_H
namespace limonp {
class InitOnOff {
public:
InitOnOff():isInited_(false) {};
~InitOnOff() {};
protected:
bool isInited_;
bool getInitFlag_()const {
return isInited_;
};
bool setInitFlag_(bool flag) {
return isInited_ = flag;
};
public:
operator bool() const {
return getInitFlag_();
};
};
}
#endif

View File

@ -1,139 +0,0 @@
#ifndef LIMONP_LOCAL_VECTOR_HPP
#define LIMONP_LOCAL_VECTOR_HPP
#include <iostream>
#include <stdlib.h>
#include <assert.h>
#include <string.h>
namespace limonp {
using namespace std;
/*
* LocalVector<T> : T must be primitive type (char , int, size_t), if T is struct or class, LocalVector<T> may be dangerous..
* LocalVector<T> is simple and not well-tested.
*/
const size_t LOCAL_VECTOR_BUFFER_SIZE = 16;
template <class T>
class LocalVector {
public:
typedef const T* const_iterator ;
typedef T value_type;
typedef size_t size_type;
private:
T buffer_[LOCAL_VECTOR_BUFFER_SIZE];
T * ptr_;
size_t size_;
size_t capacity_;
public:
LocalVector() {
init_();
};
LocalVector(const LocalVector<T>& vec) {
init_();
*this = vec;
}
LocalVector(const_iterator begin, const_iterator end) { // TODO: make it faster
init_();
while(begin != end) {
push_back(*begin++);
}
}
LocalVector(size_t size, const T& t) { // TODO: make it faster
init_();
while(size--) {
push_back(t);
}
}
~LocalVector() {
if(ptr_ != buffer_) {
free(ptr_);
}
};
public:
LocalVector<T>& operator = (const LocalVector<T>& vec) {
clear();
size_ = vec.size();
capacity_ = vec.capacity();
if(vec.buffer_ == vec.ptr_) {
memcpy(buffer_, vec.buffer_, sizeof(T) * size_);
ptr_ = buffer_;
} else {
ptr_ = (T*) malloc(vec.capacity() * sizeof(T));
assert(ptr_);
memcpy(ptr_, vec.ptr_, vec.size() * sizeof(T));
}
return *this;
}
private:
void init_() {
ptr_ = buffer_;
size_ = 0;
capacity_ = LOCAL_VECTOR_BUFFER_SIZE;
}
public:
T& operator [] (size_t i) {
return ptr_[i];
}
const T& operator [] (size_t i) const {
return ptr_[i];
}
void push_back(const T& t) {
if(size_ == capacity_) {
assert(capacity_);
reserve(capacity_ * 2);
}
ptr_[size_ ++ ] = t;
}
void reserve(size_t size) {
if(size <= capacity_) {
return;
}
T * next = (T*)malloc(sizeof(T) * size);
assert(next);
T * old = ptr_;
ptr_ = next;
memcpy(ptr_, old, sizeof(T) * capacity_);
capacity_ = size;
if(old != buffer_) {
free(old);
}
}
bool empty() const {
return 0 == size();
}
size_t size() const {
return size_;
}
size_t capacity() const {
return capacity_;
}
const_iterator begin() const {
return ptr_;
}
const_iterator end() const {
return ptr_ + size_;
}
void clear() {
if(ptr_ != buffer_) {
free(ptr_);
}
init_();
}
};
template <class T>
ostream & operator << (ostream& os, const LocalVector<T>& vec) {
if(vec.empty()) {
return os << "[]";
}
os<<"[\""<<vec[0];
for(size_t i = 1; i < vec.size(); i++) {
os<<"\", \""<<vec[i];
}
os<<"\"]";
return os;
}
}
#endif

View File

@ -1,68 +0,0 @@
/************************************
* file enc : utf8
* author : wuyanyi09@gmail.com
************************************/
#ifndef LIMONP_LOGGER_H
#define LIMONP_LOGGER_H
#include <vector>
#include <iostream>
#include <fstream>
#include <string>
#include <cstring>
#include <stdio.h>
#include <cstdlib>
#include <stdarg.h>
#include <time.h>
#include <cassert>
#define LogDebug(fmt, ...) limonp::Logger::LoggingF(limonp::LL_DEBUG, __FILE__, __LINE__, fmt, ## __VA_ARGS__)
#define LogInfo(fmt, ...) limonp::Logger::LoggingF(limonp::LL_INFO, __FILE__, __LINE__, fmt, ## __VA_ARGS__)
#define LogWarn(fmt, ...) limonp::Logger::LoggingF(limonp::LL_WARN, __FILE__, __LINE__, fmt, ## __VA_ARGS__)
#define LogError(fmt, ...) limonp::Logger::LoggingF(limonp::LL_ERROR, __FILE__, __LINE__, fmt, ## __VA_ARGS__)
#define LogFatal(fmt, ...) {limonp::Logger::LoggingF(limonp::LL_FATAL, __FILE__, __LINE__, fmt, ## __VA_ARGS__); abort();}
namespace limonp {
using namespace std;
enum {LL_DEBUG = 0, LL_INFO = 1, LL_WARN = 2, LL_ERROR = 3, LL_FATAL = 4, LEVEL_ARRAY_SIZE = 5, CSTR_BUFFER_SIZE = 32};
static const char * LOG_LEVEL_ARRAY[LEVEL_ARRAY_SIZE]= {"DEBUG","INFO","WARN","ERROR","FATAL"};
static const char * LOG_FORMAT = "%s %s:%d %s %s\n";
static const char * LOG_TIME_FORMAT = "%Y-%m-%d %H:%M:%S";
class Logger {
public:
static void Logging(size_t level, const string& msg, const char* fileName, int lineno) {
assert(level <= LL_FATAL);
char buf[CSTR_BUFFER_SIZE];
time_t timeNow;
time(&timeNow);
strftime(buf, sizeof(buf), LOG_TIME_FORMAT, localtime(&timeNow));
fprintf(stderr, LOG_FORMAT, buf, fileName, lineno,LOG_LEVEL_ARRAY[level], msg.c_str());
}
static void LoggingF(size_t level, const char* fileName, int lineno, const char* const fmt, ...) {
#ifdef LOGGER_LEVEL
if(level < LOGGER_LEVEL) return;
#endif
int size = 256;
string msg;
va_list ap;
while (1) {
msg.resize(size);
va_start(ap, fmt);
int n = vsnprintf((char *)msg.c_str(), size, fmt, ap);
va_end(ap);
if (n > -1 && n < size) {
msg.resize(n);
break;
}
if (n > -1)
size = n + 1;
else
size *= 2;
}
Logging(level, msg, fileName, lineno);
}
};
}
#endif

View File

@ -1,411 +0,0 @@
#ifndef __MD5_H__
#define __MD5_H__
// Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
// rights reserved.
// License to copy and use this software is granted provided that it
// is identified as the "RSA Data Security, Inc. MD5 Message-Digest
// Algorithm" in all material mentioning or referencing this software
// or this function.
//
// License is also granted to make and use derivative works provided
// that such works are identified as "derived from the RSA Data
// Security, Inc. MD5 Message-Digest Algorithm" in all material
// mentioning or referencing the derived work.
//
// RSA Data Security, Inc. makes no representations concerning either
// the merchantability of this software or the suitability of this
// software for any particular purpose. It is provided "as is"
// without express or implied warranty of any kind.
//
// These notices must be retained in any copies of any part of this
// documentation and/or software.
// The original md5 implementation avoids external libraries.
// This version has dependency on stdio.h for file input and
// string.h for memcpy.
#include <cstdio>
#include <cstring>
#include <iostream>
namespace limonp {
//#pragma region MD5 defines
// Constants for MD5Transform routine.
#define S11 7
#define S12 12
#define S13 17
#define S14 22
#define S21 5
#define S22 9
#define S23 14
#define S24 20
#define S31 4
#define S32 11
#define S33 16
#define S34 23
#define S41 6
#define S42 10
#define S43 15
#define S44 21
// F, G, H and I are basic MD5 functions.
#define F(x, y, z) (((x) & (y)) | ((~x) & (z)))
#define G(x, y, z) (((x) & (z)) | ((y) & (~z)))
#define H(x, y, z) ((x) ^ (y) ^ (z))
#define I(x, y, z) ((y) ^ ((x) | (~z)))
// ROTATE_LEFT rotates x left n bits.
#define ROTATE_LEFT(x, n) (((x) << (n)) | ((x) >> (32-(n))))
// FF, GG, HH, and II transformations for rounds 1, 2, 3, and 4.
// Rotation is separate from addition to prevent recomputation.
#define FF(a, b, c, d, x, s, ac) { \
(a) += F ((b), (c), (d)) + (x) + (UINT4)(ac); \
(a) = ROTATE_LEFT ((a), (s)); \
(a) += (b); \
}
#define GG(a, b, c, d, x, s, ac) { \
(a) += G ((b), (c), (d)) + (x) + (UINT4)(ac); \
(a) = ROTATE_LEFT ((a), (s)); \
(a) += (b); \
}
#define HH(a, b, c, d, x, s, ac) { \
(a) += H ((b), (c), (d)) + (x) + (UINT4)(ac); \
(a) = ROTATE_LEFT ((a), (s)); \
(a) += (b); \
}
#define II(a, b, c, d, x, s, ac) { \
(a) += I ((b), (c), (d)) + (x) + (UINT4)(ac); \
(a) = ROTATE_LEFT ((a), (s)); \
(a) += (b); \
}
//#pragma endregion
typedef unsigned char BYTE ;
// POINTER defines a generic pointer type
typedef unsigned char *POINTER;
// UINT2 defines a two byte word
typedef unsigned short int UINT2;
// UINT4 defines a four byte word
typedef unsigned int UINT4;
static unsigned char PADDING[64] = {
0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
};
// convenient object that wraps
// the C-functions for use in C++ only
class MD5 {
private:
struct __context_t {
UINT4 state[4]; /* state (ABCD) */
UINT4 count[2]; /* number of bits, modulo 2^64 (lsb first) */
unsigned char buffer[64]; /* input buffer */
} context ;
//#pragma region static helper functions
// The core of the MD5 algorithm is here.
// MD5 basic transformation. Transforms state based on block.
static void MD5Transform( UINT4 state[4], unsigned char block[64] ) {
UINT4 a = state[0], b = state[1], c = state[2], d = state[3], x[16];
Decode (x, block, 64);
/* Round 1 */
FF (a, b, c, d, x[ 0], S11, 0xd76aa478); /* 1 */
FF (d, a, b, c, x[ 1], S12, 0xe8c7b756); /* 2 */
FF (c, d, a, b, x[ 2], S13, 0x242070db); /* 3 */
FF (b, c, d, a, x[ 3], S14, 0xc1bdceee); /* 4 */
FF (a, b, c, d, x[ 4], S11, 0xf57c0faf); /* 5 */
FF (d, a, b, c, x[ 5], S12, 0x4787c62a); /* 6 */
FF (c, d, a, b, x[ 6], S13, 0xa8304613); /* 7 */
FF (b, c, d, a, x[ 7], S14, 0xfd469501); /* 8 */
FF (a, b, c, d, x[ 8], S11, 0x698098d8); /* 9 */
FF (d, a, b, c, x[ 9], S12, 0x8b44f7af); /* 10 */
FF (c, d, a, b, x[10], S13, 0xffff5bb1); /* 11 */
FF (b, c, d, a, x[11], S14, 0x895cd7be); /* 12 */
FF (a, b, c, d, x[12], S11, 0x6b901122); /* 13 */
FF (d, a, b, c, x[13], S12, 0xfd987193); /* 14 */
FF (c, d, a, b, x[14], S13, 0xa679438e); /* 15 */
FF (b, c, d, a, x[15], S14, 0x49b40821); /* 16 */
/* Round 2 */
GG (a, b, c, d, x[ 1], S21, 0xf61e2562); /* 17 */
GG (d, a, b, c, x[ 6], S22, 0xc040b340); /* 18 */
GG (c, d, a, b, x[11], S23, 0x265e5a51); /* 19 */
GG (b, c, d, a, x[ 0], S24, 0xe9b6c7aa); /* 20 */
GG (a, b, c, d, x[ 5], S21, 0xd62f105d); /* 21 */
GG (d, a, b, c, x[10], S22, 0x2441453); /* 22 */
GG (c, d, a, b, x[15], S23, 0xd8a1e681); /* 23 */
GG (b, c, d, a, x[ 4], S24, 0xe7d3fbc8); /* 24 */
GG (a, b, c, d, x[ 9], S21, 0x21e1cde6); /* 25 */
GG (d, a, b, c, x[14], S22, 0xc33707d6); /* 26 */
GG (c, d, a, b, x[ 3], S23, 0xf4d50d87); /* 27 */
GG (b, c, d, a, x[ 8], S24, 0x455a14ed); /* 28 */
GG (a, b, c, d, x[13], S21, 0xa9e3e905); /* 29 */
GG (d, a, b, c, x[ 2], S22, 0xfcefa3f8); /* 30 */
GG (c, d, a, b, x[ 7], S23, 0x676f02d9); /* 31 */
GG (b, c, d, a, x[12], S24, 0x8d2a4c8a); /* 32 */
/* Round 3 */
HH (a, b, c, d, x[ 5], S31, 0xfffa3942); /* 33 */
HH (d, a, b, c, x[ 8], S32, 0x8771f681); /* 34 */
HH (c, d, a, b, x[11], S33, 0x6d9d6122); /* 35 */
HH (b, c, d, a, x[14], S34, 0xfde5380c); /* 36 */
HH (a, b, c, d, x[ 1], S31, 0xa4beea44); /* 37 */
HH (d, a, b, c, x[ 4], S32, 0x4bdecfa9); /* 38 */
HH (c, d, a, b, x[ 7], S33, 0xf6bb4b60); /* 39 */
HH (b, c, d, a, x[10], S34, 0xbebfbc70); /* 40 */
HH (a, b, c, d, x[13], S31, 0x289b7ec6); /* 41 */
HH (d, a, b, c, x[ 0], S32, 0xeaa127fa); /* 42 */
HH (c, d, a, b, x[ 3], S33, 0xd4ef3085); /* 43 */
HH (b, c, d, a, x[ 6], S34, 0x4881d05); /* 44 */
HH (a, b, c, d, x[ 9], S31, 0xd9d4d039); /* 45 */
HH (d, a, b, c, x[12], S32, 0xe6db99e5); /* 46 */
HH (c, d, a, b, x[15], S33, 0x1fa27cf8); /* 47 */
HH (b, c, d, a, x[ 2], S34, 0xc4ac5665); /* 48 */
/* Round 4 */
II (a, b, c, d, x[ 0], S41, 0xf4292244); /* 49 */
II (d, a, b, c, x[ 7], S42, 0x432aff97); /* 50 */
II (c, d, a, b, x[14], S43, 0xab9423a7); /* 51 */
II (b, c, d, a, x[ 5], S44, 0xfc93a039); /* 52 */
II (a, b, c, d, x[12], S41, 0x655b59c3); /* 53 */
II (d, a, b, c, x[ 3], S42, 0x8f0ccc92); /* 54 */
II (c, d, a, b, x[10], S43, 0xffeff47d); /* 55 */
II (b, c, d, a, x[ 1], S44, 0x85845dd1); /* 56 */
II (a, b, c, d, x[ 8], S41, 0x6fa87e4f); /* 57 */
II (d, a, b, c, x[15], S42, 0xfe2ce6e0); /* 58 */
II (c, d, a, b, x[ 6], S43, 0xa3014314); /* 59 */
II (b, c, d, a, x[13], S44, 0x4e0811a1); /* 60 */
II (a, b, c, d, x[ 4], S41, 0xf7537e82); /* 61 */
II (d, a, b, c, x[11], S42, 0xbd3af235); /* 62 */
II (c, d, a, b, x[ 2], S43, 0x2ad7d2bb); /* 63 */
II (b, c, d, a, x[ 9], S44, 0xeb86d391); /* 64 */
state[0] += a;
state[1] += b;
state[2] += c;
state[3] += d;
// Zeroize sensitive information.
memset((POINTER)x, 0, sizeof (x));
}
// Encodes input (UINT4) into output (unsigned char). Assumes len is
// a multiple of 4.
static void Encode( unsigned char *output, UINT4 *input, unsigned int len ) {
unsigned int i, j;
for (i = 0, j = 0; j < len; i++, j += 4) {
output[j] = (unsigned char)(input[i] & 0xff);
output[j+1] = (unsigned char)((input[i] >> 8) & 0xff);
output[j+2] = (unsigned char)((input[i] >> 16) & 0xff);
output[j+3] = (unsigned char)((input[i] >> 24) & 0xff);
}
}
// Decodes input (unsigned char) into output (UINT4). Assumes len is
// a multiple of 4.
static void Decode( UINT4 *output, unsigned char *input, unsigned int len ) {
unsigned int i, j;
for (i = 0, j = 0; j < len; i++, j += 4)
output[i] = ((UINT4)input[j]) | (((UINT4)input[j+1]) << 8) |
(((UINT4)input[j+2]) << 16) | (((UINT4)input[j+3]) << 24);
}
//#pragma endregion
public:
// MAIN FUNCTIONS
MD5() {
Init() ;
}
// MD5 initialization. Begins an MD5 operation, writing a new context.
void Init() {
context.count[0] = context.count[1] = 0;
// Load magic initialization constants.
context.state[0] = 0x67452301;
context.state[1] = 0xefcdab89;
context.state[2] = 0x98badcfe;
context.state[3] = 0x10325476;
}
// MD5 block update operation. Continues an MD5 message-digest
// operation, processing another message block, and updating the
// context.
void Update(
unsigned char *input, // input block
unsigned int inputLen ) { // length of input block
unsigned int i, index, partLen;
// Compute number of bytes mod 64
index = (unsigned int)((context.count[0] >> 3) & 0x3F);
// Update number of bits
if ((context.count[0] += ((UINT4)inputLen << 3))
< ((UINT4)inputLen << 3))
context.count[1]++;
context.count[1] += ((UINT4)inputLen >> 29);
partLen = 64 - index;
// Transform as many times as possible.
if (inputLen >= partLen) {
memcpy((POINTER)&context.buffer[index], (POINTER)input, partLen);
MD5Transform (context.state, context.buffer);
for (i = partLen; i + 63 < inputLen; i += 64)
MD5Transform (context.state, &input[i]);
index = 0;
} else
i = 0;
/* Buffer remaining input */
memcpy((POINTER)&context.buffer[index], (POINTER)&input[i], inputLen-i);
}
// MD5 finalization. Ends an MD5 message-digest operation, writing the
// the message digest and zeroizing the context.
// Writes to digestRaw
void Final() {
unsigned char bits[8];
unsigned int index, padLen;
// Save number of bits
Encode( bits, context.count, 8 );
// Pad out to 56 mod 64.
index = (unsigned int)((context.count[0] >> 3) & 0x3f);
padLen = (index < 56) ? (56 - index) : (120 - index);
Update( PADDING, padLen );
// Append length (before padding)
Update( bits, 8 );
// Store state in digest
Encode( digestRaw, context.state, 16);
// Zeroize sensitive information.
memset((POINTER)&context, 0, sizeof (context));
writeToString() ;
}
/// Buffer must be 32+1 (nul) = 33 chars long at least
void writeToString() {
int pos ;
for( pos = 0 ; pos < 16 ; pos++ )
sprintf( digestChars+(pos*2), "%02x", digestRaw[pos] ) ;
}
public:
// an MD5 digest is a 16-byte number (32 hex digits)
BYTE digestRaw[ 16 ] ;
// This version of the digest is actually
// a "printf'd" version of the digest.
char digestChars[ 33 ] ;
/// Load a file from disk and digest it
// Digests a file and returns the result.
const char* digestFile( const char *filename ) {
if (NULL == filename || strcmp(filename, "") == 0)
return NULL;
Init() ;
FILE *file;
unsigned char buffer[1024] ;
if((file = fopen (filename, "rb")) == NULL) {
return NULL;
}
int len;
while( (len = fread( buffer, 1, 1024, file )) )
Update( buffer, len ) ;
Final();
fclose( file );
return digestChars ;
}
/// Digests a byte-array already in memory
const char* digestMemory( BYTE *memchunk, int len ) {
if (NULL == memchunk)
return NULL;
Init() ;
Update( memchunk, len ) ;
Final() ;
return digestChars ;
}
// Digests a string and prints the result.
const char* digestString(const char *string ) {
if (string == NULL)
return NULL;
Init() ;
Update( (unsigned char*)string, strlen(string) ) ;
Final() ;
return digestChars ;
}
};
inline bool md5String(const char* str, std::string& res) {
if (NULL == str) {
res = "";
return false;
}
MD5 md5;
const char *pRes = md5.digestString(str);
if (NULL == pRes) {
res = "";
return false;
}
res = pRes;
return true;
}
inline bool md5File(const char* filepath, std::string& res) {
if (NULL == filepath || strcmp(filepath, "") == 0) {
res = "";
return false;
}
MD5 md5;
const char *pRes = md5.digestFile(filepath);
if (NULL == pRes) {
res = "";
return false;
}
res = pRes;
return true;
}
}
#endif

View File

@ -1,47 +0,0 @@
#ifndef LIMONP_MUTEX_LOCK_HPP
#define LIMONP_MUTEX_LOCK_HPP
#include <pthread.h>
#include "NonCopyable.hpp"
#include "HandyMacro.hpp"
namespace limonp {
class MutexLock: NonCopyable {
private:
pthread_mutex_t mutex_;
public:
pthread_mutex_t* getPthreadMutex() {
return &mutex_;
}
public:
MutexLock() {
LIMONP_CHECK(!pthread_mutex_init(&mutex_, NULL));
}
~MutexLock() {
LIMONP_CHECK(!pthread_mutex_destroy(&mutex_));
}
private:
void lock() {
LIMONP_CHECK(!pthread_mutex_lock(&mutex_));
}
void unlock() {
LIMONP_CHECK(!pthread_mutex_unlock(&mutex_));
}
friend class MutexLockGuard;
};
class MutexLockGuard: NonCopyable {
public:
explicit MutexLockGuard(MutexLock & mutex)
: mutex_(mutex) {
mutex_.lock();
}
~MutexLockGuard() {
mutex_.unlock();
}
private:
MutexLock & mutex_;
};
#define MutexLockGuard(x) assert(false);
}
#endif

View File

@ -1,106 +0,0 @@
#ifndef LIMONP_MYSQLCLIENT_H
#define LIMONP_MYSQLCLIENT_H
#include <mysql.h>
#include <iostream>
#include <vector>
#include <string>
#include "Logger.hpp"
#include "InitOnOff.hpp"
#include "StringUtil.hpp"
namespace limonp {
using namespace std;
class MysqlClient: public InitOnOff {
public:
typedef vector< vector<string> > RowsType;
private:
const string host_;
const size_t port_;
const string user_;
const string passwd_;
const string db_;
const string charset_;
public:
MysqlClient(const string& host, size_t port, const string& user, const string& passwd, const string& db, const string& charset = "utf8"): host_(host), port_(port), user_(user), passwd_(passwd), db_(db), charset_(charset), conn_(NULL) {
setInitFlag_(init_());
}
~MysqlClient() {
if(conn_) {
mysql_close(conn_);
}
};
private:
bool init_() {
//cout<<mysql_get_client_info()<<endl;
if(NULL == (conn_ = mysql_init(NULL))) {
LogError("mysql_init faield. %s", mysql_error(conn_));
return false;
}
if (mysql_real_connect(conn_, host_.c_str(), user_.c_str(), passwd_.c_str(), db_.c_str(), port_, NULL, 0) == NULL) {
LogError("mysql_real_connect failed. %s", mysql_error(conn_));
mysql_close(conn_);
conn_ = NULL;
return false;
}
if(mysql_set_character_set(conn_, charset_.c_str())) {
LogError("mysql_set_character_set [%s] failed.", charset_.c_str());
return false;
}
//set reconenct
char value = 1;
mysql_options(conn_, MYSQL_OPT_RECONNECT, &value);
LogInfo("MysqlClient {host: %s, database:%s, charset:%s}", host_.c_str(), db_.c_str(), charset_.c_str());
return true;
}
public:
bool executeSql(const string& sql) {
assert(getInitFlag_());
if(mysql_query(conn_, sql.c_str())) {
LogError("mysql_query failed. %s", mysql_error(conn_));
return false;
}
return true;
}
size_t insert(const string& tableName, const string& keys, const vector<string>& vals) {
size_t retn = 0;
for(size_t i = 0; i < vals.size(); i ++) {
string sql = string_format("insert into %s (%s) values %s", tableName.c_str(), keys.c_str(), vals[i].c_str());
retn += executeSql(sql.c_str());
}
return retn;
}
bool select(const string& sql, RowsType& rows) {
if(!executeSql(sql)) {
LogError("executeSql failed. [%s]", sql.c_str());
return false;
}
MYSQL_RES * result = mysql_store_result(conn_);
if(!result) {
LogError("mysql_store_result failed.[%d]", mysql_error(conn_));
return false;
}
size_t num_fields = mysql_num_fields(result);
MYSQL_ROW row;
while((row = mysql_fetch_row(result))) {
vector<string> vec;
for(size_t i = 0; i < num_fields; i ++) {
row[i] ? vec.push_back(row[i]) : vec.push_back("NULL");
}
rows.push_back(vec);
}
mysql_free_result(result);
return true;
}
private:
MYSQL * conn_;
};
}
#endif

View File

@ -1,20 +0,0 @@
/************************************
************************************/
#ifndef LIMONP_NONCOPYABLE_H
#define LIMONP_NONCOPYABLE_H
#include <iostream>
#include <string>
namespace limonp {
class NonCopyable {
protected:
NonCopyable() {};
~NonCopyable() {};
private:
NonCopyable(const NonCopyable& );
const NonCopyable& operator=(const NonCopyable& );
};
}
#endif

View File

@ -1,145 +0,0 @@
#ifndef LIMONP_STD_EXTEMSION_HPP
#define LIMONP_STD_EXTEMSION_HPP
#include <map>
#ifdef __APPLE__
#include <unordered_map>
#include <unordered_set>
#elif(__cplusplus == 201103L)
#include <unordered_map>
#include <unordered_set>
#elif defined _MSC_VER
#include <unordered_map>
#include <unordered_set>
#else
#include <tr1/unordered_map>
#include <tr1/unordered_set>
namespace std {
using std::tr1::unordered_map;
using std::tr1::unordered_set;
}
#endif
#include <set>
#include <string>
#include <vector>
#include <deque>
#include <fstream>
#include <sstream>
namespace std {
template<typename T>
ostream& operator << (ostream& os, const vector<T>& v) {
if(v.empty()) {
return os << "[]";
}
os<<"[\""<<v[0];
for(size_t i = 1; i < v.size(); i++) {
os<<"\", \""<<v[i];
}
os<<"\"]";
return os;
}
template<typename T>
ostream& operator << (ostream& os, const deque<T>& dq) {
if(dq.empty()) {
return os << "[]";
}
os<<"[\""<<dq[0];
for(size_t i = 1; i < dq.size(); i++) {
os<<"\", \""<<dq[i];
}
os<<"\"]";
return os;
}
template<class T1, class T2>
ostream& operator << (ostream& os, const pair<T1, T2>& pr) {
os << pr.first << ":" << pr.second ;
return os;
}
template<class T>
string& operator << (string& str, const T& obj) {
stringstream ss;
ss << obj; // call ostream& operator << (ostream& os,
return str = ss.str();
}
template<class T1, class T2>
ostream& operator << (ostream& os, const map<T1, T2>& mp) {
if(mp.empty()) {
os<<"{}";
return os;
}
os<<'{';
typename map<T1, T2>::const_iterator it = mp.begin();
os<<*it;
it++;
while(it != mp.end()) {
os<<", "<<*it;
it++;
}
os<<'}';
return os;
}
template<class T1, class T2>
ostream& operator << (ostream& os, const std::unordered_map<T1, T2>& mp) {
if(mp.empty()) {
return os << "{}";
}
os<<'{';
typename std::unordered_map<T1, T2>::const_iterator it = mp.begin();
os<<*it;
it++;
while(it != mp.end()) {
os<<", "<<*it++;
}
return os<<'}';
}
template<class T>
ostream& operator << (ostream& os, const set<T>& st) {
if(st.empty()) {
os << "{}";
return os;
}
os<<'{';
typename set<T>::const_iterator it = st.begin();
os<<*it;
it++;
while(it != st.end()) {
os<<", "<<*it;
it++;
}
os<<'}';
return os;
}
template<class KeyType, class ContainType>
bool isIn(const ContainType& contain, const KeyType& key) {
return contain.end() != contain.find(key);
}
template<class T>
basic_string<T> & operator << (basic_string<T> & s, ifstream & ifs) {
return s.assign((istreambuf_iterator<T>(ifs)), istreambuf_iterator<T>());
}
template<class T>
ofstream & operator << (ofstream & ofs, const basic_string<T>& s) {
ostreambuf_iterator<T> itr (ofs);
copy(s.begin(), s.end(), itr);
return ofs;
}
} // namespace std
#endif

View File

@ -1,281 +0,0 @@
/************************************
* file enc : ascii
* author : wuyanyi09@gmail.com
************************************/
#ifndef LIMONP_STR_FUNCTS_H
#define LIMONP_STR_FUNCTS_H
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <algorithm>
#include <cctype>
#include <map>
#include <stdint.h>
#include <stdio.h>
#include <stdarg.h>
#include <memory.h>
#include <functional>
#include <locale>
#include <sstream>
#include <sys/types.h>
#include <iterator>
#include <algorithm>
#include "StdExtension.hpp"
namespace limonp {
using namespace std;
inline string string_format(const char* fmt, ...) {
int size = 256;
std::string str;
va_list ap;
while (1) {
str.resize(size);
va_start(ap, fmt);
int n = vsnprintf((char *)str.c_str(), size, fmt, ap);
va_end(ap);
if (n > -1 && n < size) {
str.resize(n);
return str;
}
if (n > -1)
size = n + 1;
else
size *= 2;
}
return str;
}
template<class T>
void join(T begin, T end, string& res, const string& connector) {
if(begin == end) {
return;
}
stringstream ss;
ss<<*begin;
begin++;
while(begin != end) {
ss << connector << *begin;
begin ++;
}
res = ss.str();
}
template<class T>
string join(T begin, T end, const string& connector) {
string res;
join(begin ,end, res, connector);
return res;
}
inline string& upper(string& str) {
transform(str.begin(), str.end(), str.begin(), (int (*)(int))toupper);
return str;
}
inline string& lower(string& str) {
transform(str.begin(), str.end(), str.begin(), (int (*)(int))tolower);
return str;
}
inline std::string& ltrim(std::string &s) {
s.erase(s.begin(), std::find_if(s.begin(), s.end(), std::not1(std::ptr_fun<int, int>(std::isspace))));
return s;
}
inline std::string& rtrim(std::string &s) {
s.erase(std::find_if(s.rbegin(), s.rend(), std::not1(std::ptr_fun<int, int>(std::isspace))).base(), s.end());
return s;
}
inline std::string& trim(std::string &s) {
return ltrim(rtrim(s));
}
inline std::string& ltrim(std::string & s, char x) {
s.erase(s.begin(), std::find_if(s.begin(), s.end(), std::not1(std::bind2nd(std::equal_to<char>(), x))));
return s;
}
inline std::string& rtrim(std::string & s, char x) {
s.erase(std::find_if(s.rbegin(), s.rend(), std::not1(std::bind2nd(std::equal_to<char>(), x))).base(), s.end());
return s;
}
inline std::string& trim(std::string &s, char x) {
return ltrim(rtrim(s, x), x);
}
inline void split(const string& src, vector<string>& res, const string& pattern, size_t maxsplit = string::npos) {
res.clear();
size_t start = 0;
size_t end = 0;
string sub;
while(start < src.size()) {
end = src.find_first_of(pattern, start);
if(string::npos == end || res.size() >= maxsplit) {
sub = src.substr(start);
trim(sub);
res.push_back(sub);
return;
}
sub = src.substr(start, end - start);
trim(sub);
res.push_back(sub);
start = end + 1;
}
return;
}
inline vector<string> split(const string& src, const string& pattern, size_t maxsplit = string::npos) {
vector<string> res;
split(src, res, pattern, maxsplit);
return res;
}
inline bool startsWith(const string& str, const string& prefix) {
if(prefix.length() > str.length()) {
return false;
}
return 0 == str.compare(0, prefix.length(), prefix);
}
inline bool endsWith(const string& str, const string& suffix) {
if(suffix.length() > str.length()) {
return false;
}
return 0 == str.compare(str.length() - suffix.length(), suffix.length(), suffix);
}
inline bool isInStr(const string& str, char ch) {
return str.find(ch) != string::npos;
}
inline uint16_t twocharToUint16(char high, char low) {
return (((uint16_t(high) & 0x00ff ) << 8) | (uint16_t(low) & 0x00ff));
}
template <class Uint16Container>
bool utf8ToUnicode(const char * const str, size_t len, Uint16Container& vec) {
if(!str) {
return false;
}
char ch1, ch2;
uint16_t tmp;
vec.clear();
for(size_t i = 0; i < len;) {
if(!(str[i] & 0x80)) { // 0xxxxxxx
vec.push_back(str[i]);
i++;
} else if ((uint8_t)str[i] <= 0xdf && i + 1 < len) { // 110xxxxxx
ch1 = (str[i] >> 2) & 0x07;
ch2 = (str[i+1] & 0x3f) | ((str[i] & 0x03) << 6 );
tmp = (((uint16_t(ch1) & 0x00ff ) << 8) | (uint16_t(ch2) & 0x00ff));
vec.push_back(tmp);
i += 2;
} else if((uint8_t)str[i] <= 0xef && i + 2 < len) {
ch1 = ((uint8_t)str[i] << 4) | ((str[i+1] >> 2) & 0x0f );
ch2 = (((uint8_t)str[i+1]<<6) & 0xc0) | (str[i+2] & 0x3f);
tmp = (((uint16_t(ch1) & 0x00ff ) << 8) | (uint16_t(ch2) & 0x00ff));
vec.push_back(tmp);
i += 3;
} else {
return false;
}
}
return true;
}
template <class Uint16Container>
bool utf8ToUnicode(const string& str, Uint16Container& vec) {
return utf8ToUnicode(str.c_str(), str.size(), vec);
}
template <class Uint16ContainerConIter>
void unicodeToUtf8(Uint16ContainerConIter begin, Uint16ContainerConIter end, string& res) {
res.clear();
uint16_t ui;
while(begin != end) {
ui = *begin;
if(ui <= 0x7f) {
res += char(ui);
} else if(ui <= 0x7ff) {
res += char(((ui>>6) & 0x1f) | 0xc0);
res += char((ui & 0x3f) | 0x80);
} else {
res += char(((ui >> 12) & 0x0f )| 0xe0);
res += char(((ui>>6) & 0x3f )| 0x80 );
res += char((ui & 0x3f) | 0x80);
}
begin ++;
}
}
template <class Uint16Container>
bool gbkTrans(const char* const str, size_t len, Uint16Container& vec) {
vec.clear();
if(!str) {
return true;
}
size_t i = 0;
while(i < len) {
if(0 == (str[i] & 0x80)) {
vec.push_back(uint16_t(str[i]));
i++;
} else {
if(i + 1 < len) { //&& (str[i+1] & 0x80))
uint16_t tmp = (((uint16_t(str[i]) & 0x00ff ) << 8) | (uint16_t(str[i+1]) & 0x00ff));
vec.push_back(tmp);
i += 2;
} else {
return false;
}
}
}
return true;
}
template <class Uint16Container>
bool gbkTrans(const string& str, Uint16Container& vec) {
return gbkTrans(str.c_str(), str.size(), vec);
}
template <class Uint16ContainerConIter>
void gbkTrans(Uint16ContainerConIter begin, Uint16ContainerConIter end, string& res) {
res.clear();
//pair<char, char> pa;
char first, second;
while(begin != end) {
//pa = uint16ToChar2(*begin);
first = ((*begin)>>8) & 0x00ff;
second = (*begin) & 0x00ff;
if(first & 0x80) {
res += first;
res += second;
} else {
res += second;
}
begin++;
}
}
/*
* format example: "%Y-%m-%d %H:%M:%S"
*/
inline void getTime(const string& format, string& timeStr) {
time_t timeNow;
time(&timeNow);
timeStr.resize(64);
size_t len = strftime((char*)timeStr.c_str(), timeStr.size(), format.c_str(), localtime(&timeNow));
timeStr.resize(len);
}
inline string pathJoin(const string& path1, const string& path2) {
if(endsWith(path1, "/")) {
return path1 + path2;
}
return path1 + "/" + path2;
}
}
#endif

View File

@ -1,42 +0,0 @@
#ifndef LIMONP_THREAD_HPP
#define LIMONP_THREAD_HPP
#include "HandyMacro.hpp"
#include "NonCopyable.hpp"
namespace limonp {
class IThread: NonCopyable {
private:
pthread_t thread_;
bool isStarted;
bool isJoined;
public:
IThread(): isStarted(false), isJoined(false) {
}
virtual ~IThread() {
if(isStarted && !isJoined) {
LIMONP_CHECK(!pthread_detach(thread_));
}
};
public:
virtual void run() = 0;
void start() {
LIMONP_CHECK(!isStarted);
LIMONP_CHECK(!pthread_create(&thread_, NULL, worker_, this));
isStarted = true;
}
void join() {
LIMONP_CHECK(!isJoined);
LIMONP_CHECK(!pthread_join(thread_, NULL));
isJoined = true;
}
private:
static void * worker_(void * data) {
IThread * ptr = (IThread* ) data;
ptr->run();
return NULL;
}
};
}
#endif

View File

@ -1,104 +0,0 @@
#ifndef LIMONP_THREAD_POOL_HPP
#define LIMONP_THREAD_POOL_HPP
#include "Thread.hpp"
#include "BlockingQueue.hpp"
namespace limonp {
class ITask {
public:
virtual void run() = 0;
virtual ~ITask() {}
};
template <class TaskType>
ITask* CreateTask() {
return new TaskType();
}
template <class TaskType, class ArgType>
ITask* CreateTask(ArgType arg) {
return new TaskType(arg);
}
template <class TaskType, class ArgType0, class ArgType1>
ITask* CreateTask(ArgType0 arg0, ArgType1 arg1) {
return new TaskType(arg0, arg1);
}
template <class TaskType, class ArgType0, class ArgType1, class ArgType2>
ITask* CreateTask(ArgType0 arg0, ArgType1 arg1, ArgType2 arg2) {
return new TaskType(arg0, arg1, arg2);
}
//class ThreadPool;
class ThreadPool: NonCopyable {
private:
class Worker: public IThread {
private:
ThreadPool * ptThreadPool_;
public:
Worker(ThreadPool* pool): ptThreadPool_(pool) {
assert(ptThreadPool_);
}
virtual ~Worker() {
}
public:
virtual void run() {
while(true) {
ITask * task = ptThreadPool_->queue_.pop();
if(task == NULL) {
break;
}
try {
task->run();
} catch(std::exception& e) {
cerr << "file:" << __FILE__
<< ", line:" << __LINE__
<< ", " << e.what() << endl;
} catch(...) {
cerr << "file:" << __FILE__
<< ", line:" << __LINE__
<< ", unknown exception." << endl;
}
delete task;
}
}
};
private:
friend class Worker;
private:
vector<IThread*> threads_;
BoundedBlockingQueue<ITask*> queue_;
//mutable MutexLock mutex_;
//Condition isEmpty__;
public:
ThreadPool(size_t threadNum, size_t queueMaxSize): threads_(threadNum), queue_(queueMaxSize) { //, mutex_(), isEmpty__(mutex_)
assert(threadNum);
assert(queueMaxSize);
for(size_t i = 0; i < threads_.size(); i ++) {
threads_[i] = new Worker(this);
}
}
~ThreadPool() {
for(size_t i = 0; i < threads_.size(); i ++) {
queue_.push(NULL);
}
for(size_t i = 0; i < threads_.size(); i ++) {
threads_[i]->join();
delete threads_[i];
}
}
public:
void start() {
for(size_t i = 0; i < threads_.size(); i++) {
threads_[i]->start();
}
}
void add(ITask* task) {
assert(task);
queue_.push(task);
}
};
}
#endif

View File

@ -1,5 +1,12 @@
SET(EXECUTABLE_OUTPUT_PATH ${PROJECT_BINARY_DIR})
ADD_EXECUTABLE(demo demo.cpp)
# Configure test paths
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/test_paths.h.in" "${CMAKE_BINARY_DIR}/test/test_paths.h")
INCLUDE_DIRECTORIES(
${CMAKE_CURRENT_BINARY_DIR}
${CMAKE_BINARY_DIR}/test
)
ADD_EXECUTABLE(load_test load_test.cpp)
ADD_SUBDIRECTORY(unittest)
ADD_SUBDIRECTORY(unittest)

View File

@ -1,60 +0,0 @@
#include "../src/Jieba.hpp"
using namespace std;
int main(int argc, char** argv) {
cppjieba::Jieba jieba("../dict/jieba.dict.utf8",
"../dict/hmm_model.utf8",
"../dict/user.dict.utf8");
vector<string> words;
string result;
string s = "我是拖拉机学院手扶拖拉机专业的。不用多久我就会升职加薪当上CEO走上人生巅峰。";
cout << "[demo] Cut With HMM" << endl;
jieba.Cut(s, words, true);
cout << limonp::join(words.begin(), words.end(), "/") << endl;
cout << "[demo] Cut Without HMM " << endl;
jieba.Cut(s, words, false);
cout << limonp::join(words.begin(), words.end(), "/") << endl;
cout << "[demo] CutAll" << endl;
jieba.CutAll(s, words);
cout << limonp::join(words.begin(), words.end(), "/") << endl;
cout << "[demo] CutForSearch" << endl;
jieba.CutForSearch(s, words);
cout << limonp::join(words.begin(), words.end(), "/") << endl;
cout << "[demo] Insert User Word" << endl;
jieba.Cut("男默女泪", words);
cout << limonp::join(words.begin(), words.end(), "/") << endl;
jieba.InsertUserWord("男默女泪");
jieba.Cut("男默女泪", words);
cout << limonp::join(words.begin(), words.end(), "/") << endl;
cout << "[demo] Locate Words" << endl;
vector<cppjieba::Jieba::LocWord> loc_words;
jieba.Cut("南京市长江大桥", words, true);
jieba.Locate(words, loc_words);
for (size_t i = 0; i < loc_words.size(); i++) {
cout << loc_words[i].word
<< ", " << loc_words[i].begin
<< ", " << loc_words[i].end
<< endl;
}
cout << "[demo] TAGGING" << endl;
vector<pair<string, string> > tagres;
jieba.Tag(s, tagres);
cout << s << endl;
cout << tagres << endl;;
//cout << "[demo] KEYWORD" << endl;
//vector<pair<string, double> > keywordres;
//jieba.Extract(s, keywordres, 5);
//cout << s << endl;
//cout << keywordres << endl;
return EXIT_SUCCESS;
}

View File

@ -1,19 +1,20 @@
#include <iostream>
#include <ctime>
#include <fstream>
#include "../src/MPSegment.hpp"
#include "../src/HMMSegment.hpp"
#include "../src/MixSegment.hpp"
#include "../src/KeywordExtractor.hpp"
#include "../src/limonp/Colors.hpp"
#include "cppjieba/MPSegment.hpp"
#include "cppjieba/HMMSegment.hpp"
#include "cppjieba/MixSegment.hpp"
#include "cppjieba/KeywordExtractor.hpp"
#include "limonp/Colors.hpp"
#include "test_paths.h"
using namespace cppjieba;
void Cut(size_t times = 50) {
MixSegment seg("../dict/jieba.dict.utf8", "../dict/hmm_model.utf8");
MixSegment seg(DICT_DIR "/jieba.dict.utf8", DICT_DIR "/hmm_model.utf8");
vector<string> res;
string doc;
ifstream ifs("../test/testdata/weicheng.utf8");
ifstream ifs(TEST_DATA_DIR "/weicheng.utf8");
assert(ifs);
doc << ifs;
long beginTime = clock();
@ -29,10 +30,13 @@ void Cut(size_t times = 50) {
}
void Extract(size_t times = 400) {
KeywordExtractor Extractor("../dict/jieba.dict.utf8", "../dict/hmm_model.utf8", "../dict/idf.utf8", "../dict/stop_words.utf8");
KeywordExtractor Extractor(DICT_DIR "/jieba.dict.utf8",
DICT_DIR "/hmm_model.utf8",
DICT_DIR "/idf.utf8",
DICT_DIR "/stop_words.utf8");
vector<string> words;
string doc;
ifstream ifs("../test/testdata/review.100");
ifstream ifs(TEST_DATA_DIR "/review.100");
assert(ifs);
doc << ifs;
long beginTime = clock();
@ -51,4 +55,4 @@ int main(int argc, char ** argv) {
Cut();
Extract();
return EXIT_SUCCESS;
}
}

View File

@ -1,2 +0,0 @@
# go get github.com/yanyiwu/go_http_load
go_http_load -method=GET -get_urls="../test/testdata/load_test.urls" -loop_count=500 -goroutines=2

View File

@ -1,91 +0,0 @@
#!/usr/bin/python
# coding:utf-8
import time
import urllib2
import threading
from Queue import Queue
from time import sleep
import sys
# 性能测试页面
#PERF_TEST_URL = "http://10.2.66.38/?yyid=-1&suv=1309231700203264&callback=xxxxx"
URLS = [line for line in open("../test/testdata/load_test.urls", "r")]
# 配置:压力测试
THREAD_NUM = 10 # 并发线程总数
ONE_WORKER_NUM = 500 # 每个线程的循环次数
LOOP_SLEEP = 0.01 # 每次请求时间间隔(秒)
# 配置:模拟运行状态
#THREAD_NUM = 10 # 并发线程总数
#ONE_WORKER_NUM = 10 # 每个线程的循环次数
#LOOP_SLEEP = 0 # 每次请求时间间隔(秒)
# 出错数
ERROR_NUM = 0
#具体的处理函数,负责处理单个任务
def doWork(index, url):
t = threading.currentThread()
#print "["+t.name+" "+str(index)+"] "+PERF_TEST_URL
try:
html = urllib2.urlopen(url).read()
except urllib2.URLError, e:
print "["+t.name+" "+str(index)+"] "
print e
global ERROR_NUM
ERROR_NUM += 1
#这个是工作进程,负责不断从队列取数据并处理
def working():
t = threading.currentThread()
print "["+t.name+"] Sub Thread Begin"
i = 0
while i < ONE_WORKER_NUM:
i += 1
doWork(i, URLS[i % len(URLS)])
sleep(LOOP_SLEEP)
print "["+t.name+"] Sub Thread End"
def main():
#doWork(0)
#return
t1 = time.time()
Threads = []
# 创建线程
for i in range(THREAD_NUM):
t = threading.Thread(target=working, name="T"+str(i))
t.setDaemon(True)
Threads.append(t)
for t in Threads:
t.start()
for t in Threads:
t.join()
print "main thread end"
t2 = time.time()
print "========================================"
#print "URL:", PERF_TEST_URL
print "任务数量:", THREAD_NUM, "*", ONE_WORKER_NUM, "=", THREAD_NUM*ONE_WORKER_NUM
print "总耗时(秒):", t2-t1
print "每次请求耗时(秒):", (t2-t1) / (THREAD_NUM*ONE_WORKER_NUM)
print "每秒承载请求数:", 1 / ((t2-t1) / (THREAD_NUM*ONE_WORKER_NUM))
print "错误数量:", ERROR_NUM
if __name__ == "__main__":
main()

View File

@ -1,11 +0,0 @@
CURL_RES=../test/testdata/curl.res
TMP=curl.res.tmp
curl -s "http://127.0.0.1:11200/?key=南京市长江大桥" >> $TMP
if diff $TMP $CURL_RES >> /dev/null
then
echo "ok";
else
echo "failed."
fi
rm $TMP

7
test/test_paths.h.in Normal file
View File

@ -0,0 +1,7 @@
#ifndef TEST_PATHS_H
#define TEST_PATHS_H
#define TEST_DATA_DIR "@CMAKE_CURRENT_SOURCE_DIR@/testdata"
#define DICT_DIR "@CMAKE_SOURCE_DIR@/dict"
#endif // TEST_PATHS_H

View File

@ -5,3 +5,4 @@ B
iPhone6
蓝翔 nz
忽如一夜春风来
区块链 10 nz

View File

@ -1,11 +1,30 @@
message(STATUS "MSVC value: ${MSVC}")
if (MSVC)
set(CMAKE_MSVC_RUNTIME_LIBRARY "MultiThreadedDebugDLL")
set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
add_compile_options(/utf-8)
endif()
include(FetchContent)
FetchContent_Declare(
googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG release-1.12.1
)
FetchContent_MakeAvailable(googletest)
SET(EXECUTABLE_OUTPUT_PATH ${PROJECT_BINARY_DIR}/test)
SET(LIBRARY_OUTPUT_PATH ${PROJECT_BINARY_DIR}/lib)
SET(GTEST_ROOT_DIR gtest)
ADD_DEFINITIONS(-DLOGGING_LEVEL=LL_WARNING)
ADD_DEFINITIONS(-DLOGGER_LEVEL=LL_WARN)
INCLUDE_DIRECTORIES(${GTEST_ROOT_DIR} ${GTEST_ROOT_DIR}/include ${PROJECT_SOURCE_DIR})
ADD_LIBRARY(gtest STATIC ${GTEST_ROOT_DIR}/src/gtest-all.cc)
# Add include directories
INCLUDE_DIRECTORIES(
${CMAKE_CURRENT_BINARY_DIR}
${CMAKE_BINARY_DIR}/test
)
ADD_EXECUTABLE(test.run
gtest_main.cpp
@ -15,7 +34,8 @@ ADD_EXECUTABLE(test.run
pos_tagger_test.cpp
jieba_test.cpp
pre_filter_test.cpp
unicode_test.cpp
textrank_test.cpp
)
TARGET_LINK_LIBRARIES(gtest pthread)
TARGET_LINK_LIBRARIES(test.run gtest pthread)
TARGET_LINK_LIBRARIES(test.run gtest)

View File

@ -1,283 +0,0 @@
// Copyright 2005, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Author: wan@google.com (Zhanyong Wan)
//
// The Google C++ Testing Framework (Google Test)
//
// This header file defines the public API for death tests. It is
// #included by gtest.h so a user doesn't need to include this
// directly.
#ifndef GTEST_INCLUDE_GTEST_GTEST_DEATH_TEST_H_
#define GTEST_INCLUDE_GTEST_GTEST_DEATH_TEST_H_
#include "gtest/internal/gtest-death-test-internal.h"
namespace testing {
// This flag controls the style of death tests. Valid values are "threadsafe",
// meaning that the death test child process will re-execute the test binary
// from the start, running only a single death test, or "fast",
// meaning that the child process will execute the test logic immediately
// after forking.
GTEST_DECLARE_string_(death_test_style);
#if GTEST_HAS_DEATH_TEST
// The following macros are useful for writing death tests.
// Here's what happens when an ASSERT_DEATH* or EXPECT_DEATH* is
// executed:
//
// 1. It generates a warning if there is more than one active
// thread. This is because it's safe to fork() or clone() only
// when there is a single thread.
//
// 2. The parent process clone()s a sub-process and runs the death
// test in it; the sub-process exits with code 0 at the end of the
// death test, if it hasn't exited already.
//
// 3. The parent process waits for the sub-process to terminate.
//
// 4. The parent process checks the exit code and error message of
// the sub-process.
//
// Examples:
//
// ASSERT_DEATH(server.SendMessage(56, "Hello"), "Invalid port number");
// for (int i = 0; i < 5; i++) {
// EXPECT_DEATH(server.ProcessRequest(i),
// "Invalid request .* in ProcessRequest()")
// << "Failed to die on request " << i);
// }
//
// ASSERT_EXIT(server.ExitNow(), ::testing::ExitedWithCode(0), "Exiting");
//
// bool KilledBySIGHUP(int exit_code) {
// return WIFSIGNALED(exit_code) && WTERMSIG(exit_code) == SIGHUP;
// }
//
// ASSERT_EXIT(client.HangUpServer(), KilledBySIGHUP, "Hanging up!");
//
// On the regular expressions used in death tests:
//
// On POSIX-compliant systems (*nix), we use the <regex.h> library,
// which uses the POSIX extended regex syntax.
//
// On other platforms (e.g. Windows), we only support a simple regex
// syntax implemented as part of Google Test. This limited
// implementation should be enough most of the time when writing
// death tests; though it lacks many features you can find in PCRE
// or POSIX extended regex syntax. For example, we don't support
// union ("x|y"), grouping ("(xy)"), brackets ("[xy]"), and
// repetition count ("x{5,7}"), among others.
//
// Below is the syntax that we do support. We chose it to be a
// subset of both PCRE and POSIX extended regex, so it's easy to
// learn wherever you come from. In the following: 'A' denotes a
// literal character, period (.), or a single \\ escape sequence;
// 'x' and 'y' denote regular expressions; 'm' and 'n' are for
// natural numbers.
//
// c matches any literal character c
// \\d matches any decimal digit
// \\D matches any character that's not a decimal digit
// \\f matches \f
// \\n matches \n
// \\r matches \r
// \\s matches any ASCII whitespace, including \n
// \\S matches any character that's not a whitespace
// \\t matches \t
// \\v matches \v
// \\w matches any letter, _, or decimal digit
// \\W matches any character that \\w doesn't match
// \\c matches any literal character c, which must be a punctuation
// . matches any single character except \n
// A? matches 0 or 1 occurrences of A
// A* matches 0 or many occurrences of A
// A+ matches 1 or many occurrences of A
// ^ matches the beginning of a string (not that of each line)
// $ matches the end of a string (not that of each line)
// xy matches x followed by y
//
// If you accidentally use PCRE or POSIX extended regex features
// not implemented by us, you will get a run-time failure. In that
// case, please try to rewrite your regular expression within the
// above syntax.
//
// This implementation is *not* meant to be as highly tuned or robust
// as a compiled regex library, but should perform well enough for a
// death test, which already incurs significant overhead by launching
// a child process.
//
// Known caveats:
//
// A "threadsafe" style death test obtains the path to the test
// program from argv[0] and re-executes it in the sub-process. For
// simplicity, the current implementation doesn't search the PATH
// when launching the sub-process. This means that the user must
// invoke the test program via a path that contains at least one
// path separator (e.g. path/to/foo_test and
// /absolute/path/to/bar_test are fine, but foo_test is not). This
// is rarely a problem as people usually don't put the test binary
// directory in PATH.
//
// TODO(wan@google.com): make thread-safe death tests search the PATH.
// Asserts that a given statement causes the program to exit, with an
// integer exit status that satisfies predicate, and emitting error output
// that matches regex.
# define ASSERT_EXIT(statement, predicate, regex) \
GTEST_DEATH_TEST_(statement, predicate, regex, GTEST_FATAL_FAILURE_)
// Like ASSERT_EXIT, but continues on to successive tests in the
// test case, if any:
# define EXPECT_EXIT(statement, predicate, regex) \
GTEST_DEATH_TEST_(statement, predicate, regex, GTEST_NONFATAL_FAILURE_)
// Asserts that a given statement causes the program to exit, either by
// explicitly exiting with a nonzero exit code or being killed by a
// signal, and emitting error output that matches regex.
# define ASSERT_DEATH(statement, regex) \
ASSERT_EXIT(statement, ::testing::internal::ExitedUnsuccessfully, regex)
// Like ASSERT_DEATH, but continues on to successive tests in the
// test case, if any:
# define EXPECT_DEATH(statement, regex) \
EXPECT_EXIT(statement, ::testing::internal::ExitedUnsuccessfully, regex)
// Two predicate classes that can be used in {ASSERT,EXPECT}_EXIT*:
// Tests that an exit code describes a normal exit with a given exit code.
class GTEST_API_ ExitedWithCode {
public:
explicit ExitedWithCode(int exit_code);
bool operator()(int exit_status) const;
private:
// No implementation - assignment is unsupported.
void operator=(const ExitedWithCode& other);
const int exit_code_;
};
# if !GTEST_OS_WINDOWS
// Tests that an exit code describes an exit due to termination by a
// given signal.
class GTEST_API_ KilledBySignal {
public:
explicit KilledBySignal(int signum);
bool operator()(int exit_status) const;
private:
const int signum_;
};
# endif // !GTEST_OS_WINDOWS
// EXPECT_DEBUG_DEATH asserts that the given statements die in debug mode.
// The death testing framework causes this to have interesting semantics,
// since the sideeffects of the call are only visible in opt mode, and not
// in debug mode.
//
// In practice, this can be used to test functions that utilize the
// LOG(DFATAL) macro using the following style:
//
// int DieInDebugOr12(int* sideeffect) {
// if (sideeffect) {
// *sideeffect = 12;
// }
// LOG(DFATAL) << "death";
// return 12;
// }
//
// TEST(TestCase, TestDieOr12WorksInDgbAndOpt) {
// int sideeffect = 0;
// // Only asserts in dbg.
// EXPECT_DEBUG_DEATH(DieInDebugOr12(&sideeffect), "death");
//
// #ifdef NDEBUG
// // opt-mode has sideeffect visible.
// EXPECT_EQ(12, sideeffect);
// #else
// // dbg-mode no visible sideeffect.
// EXPECT_EQ(0, sideeffect);
// #endif
// }
//
// This will assert that DieInDebugReturn12InOpt() crashes in debug
// mode, usually due to a DCHECK or LOG(DFATAL), but returns the
// appropriate fallback value (12 in this case) in opt mode. If you
// need to test that a function has appropriate side-effects in opt
// mode, include assertions against the side-effects. A general
// pattern for this is:
//
// EXPECT_DEBUG_DEATH({
// // Side-effects here will have an effect after this statement in
// // opt mode, but none in debug mode.
// EXPECT_EQ(12, DieInDebugOr12(&sideeffect));
// }, "death");
//
# ifdef NDEBUG
# define EXPECT_DEBUG_DEATH(statement, regex) \
do { statement; } while (::testing::internal::AlwaysFalse())
# define ASSERT_DEBUG_DEATH(statement, regex) \
do { statement; } while (::testing::internal::AlwaysFalse())
# else
# define EXPECT_DEBUG_DEATH(statement, regex) \
EXPECT_DEATH(statement, regex)
# define ASSERT_DEBUG_DEATH(statement, regex) \
ASSERT_DEATH(statement, regex)
# endif // NDEBUG for EXPECT_DEBUG_DEATH
#endif // GTEST_HAS_DEATH_TEST
// EXPECT_DEATH_IF_SUPPORTED(statement, regex) and
// ASSERT_DEATH_IF_SUPPORTED(statement, regex) expand to real death tests if
// death tests are supported; otherwise they just issue a warning. This is
// useful when you are combining death test assertions with normal test
// assertions in one test.
#if GTEST_HAS_DEATH_TEST
# define EXPECT_DEATH_IF_SUPPORTED(statement, regex) \
EXPECT_DEATH(statement, regex)
# define ASSERT_DEATH_IF_SUPPORTED(statement, regex) \
ASSERT_DEATH(statement, regex)
#else
# define EXPECT_DEATH_IF_SUPPORTED(statement, regex) \
GTEST_UNSUPPORTED_DEATH_TEST_(statement, regex, )
# define ASSERT_DEATH_IF_SUPPORTED(statement, regex) \
GTEST_UNSUPPORTED_DEATH_TEST_(statement, regex, return)
#endif
} // namespace testing
#endif // GTEST_INCLUDE_GTEST_GTEST_DEATH_TEST_H_

View File

@ -1,230 +0,0 @@
// Copyright 2005, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Author: wan@google.com (Zhanyong Wan)
//
// The Google C++ Testing Framework (Google Test)
//
// This header file defines the Message class.
//
// IMPORTANT NOTE: Due to limitation of the C++ language, we have to
// leave some internal implementation details in this header file.
// They are clearly marked by comments like this:
//
// // INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM.
//
// Such code is NOT meant to be used by a user directly, and is subject
// to CHANGE WITHOUT NOTICE. Therefore DO NOT DEPEND ON IT in a user
// program!
#ifndef GTEST_INCLUDE_GTEST_GTEST_MESSAGE_H_
#define GTEST_INCLUDE_GTEST_GTEST_MESSAGE_H_
#include <limits>
#include "gtest/internal/gtest-string.h"
#include "gtest/internal/gtest-internal.h"
namespace testing {
// The Message class works like an ostream repeater.
//
// Typical usage:
//
// 1. You stream a bunch of values to a Message object.
// It will remember the text in a stringstream.
// 2. Then you stream the Message object to an ostream.
// This causes the text in the Message to be streamed
// to the ostream.
//
// For example;
//
// testing::Message foo;
// foo << 1 << " != " << 2;
// std::cout << foo;
//
// will print "1 != 2".
//
// Message is not intended to be inherited from. In particular, its
// destructor is not virtual.
//
// Note that stringstream behaves differently in gcc and in MSVC. You
// can stream a NULL char pointer to it in the former, but not in the
// latter (it causes an access violation if you do). The Message
// class hides this difference by treating a NULL char pointer as
// "(null)".
class GTEST_API_ Message {
private:
// The type of basic IO manipulators (endl, ends, and flush) for
// narrow streams.
typedef std::ostream& (*BasicNarrowIoManip)(std::ostream&);
public:
// Constructs an empty Message.
// We allocate the stringstream separately because otherwise each use of
// ASSERT/EXPECT in a procedure adds over 200 bytes to the procedure's
// stack frame leading to huge stack frames in some cases; gcc does not reuse
// the stack space.
Message() : ss_(new ::std::stringstream) {
// By default, we want there to be enough precision when printing
// a double to a Message.
*ss_ << std::setprecision(std::numeric_limits<double>::digits10 + 2);
}
// Copy constructor.
Message(const Message& msg) : ss_(new ::std::stringstream) { // NOLINT
*ss_ << msg.GetString();
}
// Constructs a Message from a C-string.
explicit Message(const char* str) : ss_(new ::std::stringstream) {
*ss_ << str;
}
#if GTEST_OS_SYMBIAN
// Streams a value (either a pointer or not) to this object.
template <typename T>
inline Message& operator <<(const T& value) {
StreamHelper(typename internal::is_pointer<T>::type(), value);
return *this;
}
#else
// Streams a non-pointer value to this object.
template <typename T>
inline Message& operator <<(const T& val) {
::GTestStreamToHelper(ss_.get(), val);
return *this;
}
// Streams a pointer value to this object.
//
// This function is an overload of the previous one. When you
// stream a pointer to a Message, this definition will be used as it
// is more specialized. (The C++ Standard, section
// [temp.func.order].) If you stream a non-pointer, then the
// previous definition will be used.
//
// The reason for this overload is that streaming a NULL pointer to
// ostream is undefined behavior. Depending on the compiler, you
// may get "0", "(nil)", "(null)", or an access violation. To
// ensure consistent result across compilers, we always treat NULL
// as "(null)".
template <typename T>
inline Message& operator <<(T* const& pointer) { // NOLINT
if (pointer == NULL) {
*ss_ << "(null)";
} else {
::GTestStreamToHelper(ss_.get(), pointer);
}
return *this;
}
#endif // GTEST_OS_SYMBIAN
// Since the basic IO manipulators are overloaded for both narrow
// and wide streams, we have to provide this specialized definition
// of operator <<, even though its body is the same as the
// templatized version above. Without this definition, streaming
// endl or other basic IO manipulators to Message will confuse the
// compiler.
Message& operator <<(BasicNarrowIoManip val) {
*ss_ << val;
return *this;
}
// Instead of 1/0, we want to see true/false for bool values.
Message& operator <<(bool b) {
return *this << (b ? "true" : "false");
}
// These two overloads allow streaming a wide C string to a Message
// using the UTF-8 encoding.
Message& operator <<(const wchar_t* wide_c_str) {
return *this << internal::String::ShowWideCString(wide_c_str);
}
Message& operator <<(wchar_t* wide_c_str) {
return *this << internal::String::ShowWideCString(wide_c_str);
}
#if GTEST_HAS_STD_WSTRING
// Converts the given wide string to a narrow string using the UTF-8
// encoding, and streams the result to this Message object.
Message& operator <<(const ::std::wstring& wstr);
#endif // GTEST_HAS_STD_WSTRING
#if GTEST_HAS_GLOBAL_WSTRING
// Converts the given wide string to a narrow string using the UTF-8
// encoding, and streams the result to this Message object.
Message& operator <<(const ::wstring& wstr);
#endif // GTEST_HAS_GLOBAL_WSTRING
// Gets the text streamed to this object so far as a String.
// Each '\0' character in the buffer is replaced with "\\0".
//
// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM.
internal::String GetString() const {
return internal::StringStreamToString(ss_.get());
}
private:
#if GTEST_OS_SYMBIAN
// These are needed as the Nokia Symbian Compiler cannot decide between
// const T& and const T* in a function template. The Nokia compiler _can_
// decide between class template specializations for T and T*, so a
// tr1::type_traits-like is_pointer works, and we can overload on that.
template <typename T>
inline void StreamHelper(internal::true_type /*dummy*/, T* pointer) {
if (pointer == NULL) {
*ss_ << "(null)";
} else {
::GTestStreamToHelper(ss_.get(), pointer);
}
}
template <typename T>
inline void StreamHelper(internal::false_type /*dummy*/, const T& value) {
::GTestStreamToHelper(ss_.get(), value);
}
#endif // GTEST_OS_SYMBIAN
// We'll hold the text streamed to this object here.
const internal::scoped_ptr< ::std::stringstream> ss_;
// We declare (but don't implement) this to prevent the compiler
// from implementing the assignment operator.
void operator=(const Message&);
};
// Streams a Message to an ostream.
inline std::ostream& operator <<(std::ostream& os, const Message& sb) {
return os << sb.GetString();
}
} // namespace testing
#endif // GTEST_INCLUDE_GTEST_GTEST_MESSAGE_H_

File diff suppressed because it is too large Load Diff

View File

@ -1,487 +0,0 @@
$$ -*- mode: c++; -*-
$var n = 50 $$ Maximum length of Values arguments we want to support.
$var maxtuple = 10 $$ Maximum number of Combine arguments we want to support.
// Copyright 2008, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Authors: vladl@google.com (Vlad Losev)
//
// Macros and functions for implementing parameterized tests
// in Google C++ Testing Framework (Google Test)
//
// This file is generated by a SCRIPT. DO NOT EDIT BY HAND!
//
#ifndef GTEST_INCLUDE_GTEST_GTEST_PARAM_TEST_H_
#define GTEST_INCLUDE_GTEST_GTEST_PARAM_TEST_H_
// Value-parameterized tests allow you to test your code with different
// parameters without writing multiple copies of the same test.
//
// Here is how you use value-parameterized tests:
#if 0
// To write value-parameterized tests, first you should define a fixture
// class. It is usually derived from testing::TestWithParam<T> (see below for
// another inheritance scheme that's sometimes useful in more complicated
// class hierarchies), where the type of your parameter values.
// TestWithParam<T> is itself derived from testing::Test. T can be any
// copyable type. If it's a raw pointer, you are responsible for managing the
// lifespan of the pointed values.
class FooTest : public ::testing::TestWithParam<const char*> {
// You can implement all the usual class fixture members here.
};
// Then, use the TEST_P macro to define as many parameterized tests
// for this fixture as you want. The _P suffix is for "parameterized"
// or "pattern", whichever you prefer to think.
TEST_P(FooTest, DoesBlah) {
// Inside a test, access the test parameter with the GetParam() method
// of the TestWithParam<T> class:
EXPECT_TRUE(foo.Blah(GetParam()));
...
}
TEST_P(FooTest, HasBlahBlah) {
...
}
// Finally, you can use INSTANTIATE_TEST_CASE_P to instantiate the test
// case with any set of parameters you want. Google Test defines a number
// of functions for generating test parameters. They return what we call
// (surprise!) parameter generators. Here is a summary of them, which
// are all in the testing namespace:
//
//
// Range(begin, end [, step]) - Yields values {begin, begin+step,
// begin+step+step, ...}. The values do not
// include end. step defaults to 1.
// Values(v1, v2, ..., vN) - Yields values {v1, v2, ..., vN}.
// ValuesIn(container) - Yields values from a C-style array, an STL
// ValuesIn(begin,end) container, or an iterator range [begin, end).
// Bool() - Yields sequence {false, true}.
// Combine(g1, g2, ..., gN) - Yields all combinations (the Cartesian product
// for the math savvy) of the values generated
// by the N generators.
//
// For more details, see comments at the definitions of these functions below
// in this file.
//
// The following statement will instantiate tests from the FooTest test case
// each with parameter values "meeny", "miny", and "moe".
INSTANTIATE_TEST_CASE_P(InstantiationName,
FooTest,
Values("meeny", "miny", "moe"));
// To distinguish different instances of the pattern, (yes, you
// can instantiate it more then once) the first argument to the
// INSTANTIATE_TEST_CASE_P macro is a prefix that will be added to the
// actual test case name. Remember to pick unique prefixes for different
// instantiations. The tests from the instantiation above will have
// these names:
//
// * InstantiationName/FooTest.DoesBlah/0 for "meeny"
// * InstantiationName/FooTest.DoesBlah/1 for "miny"
// * InstantiationName/FooTest.DoesBlah/2 for "moe"
// * InstantiationName/FooTest.HasBlahBlah/0 for "meeny"
// * InstantiationName/FooTest.HasBlahBlah/1 for "miny"
// * InstantiationName/FooTest.HasBlahBlah/2 for "moe"
//
// You can use these names in --gtest_filter.
//
// This statement will instantiate all tests from FooTest again, each
// with parameter values "cat" and "dog":
const char* pets[] = {"cat", "dog"};
INSTANTIATE_TEST_CASE_P(AnotherInstantiationName, FooTest, ValuesIn(pets));
// The tests from the instantiation above will have these names:
//
// * AnotherInstantiationName/FooTest.DoesBlah/0 for "cat"
// * AnotherInstantiationName/FooTest.DoesBlah/1 for "dog"
// * AnotherInstantiationName/FooTest.HasBlahBlah/0 for "cat"
// * AnotherInstantiationName/FooTest.HasBlahBlah/1 for "dog"
//
// Please note that INSTANTIATE_TEST_CASE_P will instantiate all tests
// in the given test case, whether their definitions come before or
// AFTER the INSTANTIATE_TEST_CASE_P statement.
//
// Please also note that generator expressions (including parameters to the
// generators) are evaluated in InitGoogleTest(), after main() has started.
// This allows the user on one hand, to adjust generator parameters in order
// to dynamically determine a set of tests to run and on the other hand,
// give the user a chance to inspect the generated tests with Google Test
// reflection API before RUN_ALL_TESTS() is executed.
//
// You can see samples/sample7_unittest.cc and samples/sample8_unittest.cc
// for more examples.
//
// In the future, we plan to publish the API for defining new parameter
// generators. But for now this interface remains part of the internal
// implementation and is subject to change.
//
//
// A parameterized test fixture must be derived from testing::Test and from
// testing::WithParamInterface<T>, where T is the type of the parameter
// values. Inheriting from TestWithParam<T> satisfies that requirement because
// TestWithParam<T> inherits from both Test and WithParamInterface. In more
// complicated hierarchies, however, it is occasionally useful to inherit
// separately from Test and WithParamInterface. For example:
class BaseTest : public ::testing::Test {
// You can inherit all the usual members for a non-parameterized test
// fixture here.
};
class DerivedTest : public BaseTest, public ::testing::WithParamInterface<int> {
// The usual test fixture members go here too.
};
TEST_F(BaseTest, HasFoo) {
// This is an ordinary non-parameterized test.
}
TEST_P(DerivedTest, DoesBlah) {
// GetParam works just the same here as if you inherit from TestWithParam.
EXPECT_TRUE(foo.Blah(GetParam()));
}
#endif // 0
#include "gtest/internal/gtest-port.h"
#if !GTEST_OS_SYMBIAN
# include <utility>
#endif
// scripts/fuse_gtest.py depends on gtest's own header being #included
// *unconditionally*. Therefore these #includes cannot be moved
// inside #if GTEST_HAS_PARAM_TEST.
#include "gtest/internal/gtest-internal.h"
#include "gtest/internal/gtest-param-util.h"
#include "gtest/internal/gtest-param-util-generated.h"
#if GTEST_HAS_PARAM_TEST
namespace testing {
// Functions producing parameter generators.
//
// Google Test uses these generators to produce parameters for value-
// parameterized tests. When a parameterized test case is instantiated
// with a particular generator, Google Test creates and runs tests
// for each element in the sequence produced by the generator.
//
// In the following sample, tests from test case FooTest are instantiated
// each three times with parameter values 3, 5, and 8:
//
// class FooTest : public TestWithParam<int> { ... };
//
// TEST_P(FooTest, TestThis) {
// }
// TEST_P(FooTest, TestThat) {
// }
// INSTANTIATE_TEST_CASE_P(TestSequence, FooTest, Values(3, 5, 8));
//
// Range() returns generators providing sequences of values in a range.
//
// Synopsis:
// Range(start, end)
// - returns a generator producing a sequence of values {start, start+1,
// start+2, ..., }.
// Range(start, end, step)
// - returns a generator producing a sequence of values {start, start+step,
// start+step+step, ..., }.
// Notes:
// * The generated sequences never include end. For example, Range(1, 5)
// returns a generator producing a sequence {1, 2, 3, 4}. Range(1, 9, 2)
// returns a generator producing {1, 3, 5, 7}.
// * start and end must have the same type. That type may be any integral or
// floating-point type or a user defined type satisfying these conditions:
// * It must be assignable (have operator=() defined).
// * It must have operator+() (operator+(int-compatible type) for
// two-operand version).
// * It must have operator<() defined.
// Elements in the resulting sequences will also have that type.
// * Condition start < end must be satisfied in order for resulting sequences
// to contain any elements.
//
template <typename T, typename IncrementT>
internal::ParamGenerator<T> Range(T start, T end, IncrementT step) {
return internal::ParamGenerator<T>(
new internal::RangeGenerator<T, IncrementT>(start, end, step));
}
template <typename T>
internal::ParamGenerator<T> Range(T start, T end) {
return Range(start, end, 1);
}
// ValuesIn() function allows generation of tests with parameters coming from
// a container.
//
// Synopsis:
// ValuesIn(const T (&array)[N])
// - returns a generator producing sequences with elements from
// a C-style array.
// ValuesIn(const Container& container)
// - returns a generator producing sequences with elements from
// an STL-style container.
// ValuesIn(Iterator begin, Iterator end)
// - returns a generator producing sequences with elements from
// a range [begin, end) defined by a pair of STL-style iterators. These
// iterators can also be plain C pointers.
//
// Please note that ValuesIn copies the values from the containers
// passed in and keeps them to generate tests in RUN_ALL_TESTS().
//
// Examples:
//
// This instantiates tests from test case StringTest
// each with C-string values of "foo", "bar", and "baz":
//
// const char* strings[] = {"foo", "bar", "baz"};
// INSTANTIATE_TEST_CASE_P(StringSequence, SrtingTest, ValuesIn(strings));
//
// This instantiates tests from test case StlStringTest
// each with STL strings with values "a" and "b":
//
// ::std::vector< ::std::string> GetParameterStrings() {
// ::std::vector< ::std::string> v;
// v.push_back("a");
// v.push_back("b");
// return v;
// }
//
// INSTANTIATE_TEST_CASE_P(CharSequence,
// StlStringTest,
// ValuesIn(GetParameterStrings()));
//
//
// This will also instantiate tests from CharTest
// each with parameter values 'a' and 'b':
//
// ::std::list<char> GetParameterChars() {
// ::std::list<char> list;
// list.push_back('a');
// list.push_back('b');
// return list;
// }
// ::std::list<char> l = GetParameterChars();
// INSTANTIATE_TEST_CASE_P(CharSequence2,
// CharTest,
// ValuesIn(l.begin(), l.end()));
//
template <typename ForwardIterator>
internal::ParamGenerator<
typename ::testing::internal::IteratorTraits<ForwardIterator>::value_type>
ValuesIn(ForwardIterator begin, ForwardIterator end) {
typedef typename ::testing::internal::IteratorTraits<ForwardIterator>
::value_type ParamType;
return internal::ParamGenerator<ParamType>(
new internal::ValuesInIteratorRangeGenerator<ParamType>(begin, end));
}
template <typename T, size_t N>
internal::ParamGenerator<T> ValuesIn(const T (&array)[N]) {
return ValuesIn(array, array + N);
}
template <class Container>
internal::ParamGenerator<typename Container::value_type> ValuesIn(
const Container& container) {
return ValuesIn(container.begin(), container.end());
}
// Values() allows generating tests from explicitly specified list of
// parameters.
//
// Synopsis:
// Values(T v1, T v2, ..., T vN)
// - returns a generator producing sequences with elements v1, v2, ..., vN.
//
// For example, this instantiates tests from test case BarTest each
// with values "one", "two", and "three":
//
// INSTANTIATE_TEST_CASE_P(NumSequence, BarTest, Values("one", "two", "three"));
//
// This instantiates tests from test case BazTest each with values 1, 2, 3.5.
// The exact type of values will depend on the type of parameter in BazTest.
//
// INSTANTIATE_TEST_CASE_P(FloatingNumbers, BazTest, Values(1, 2, 3.5));
//
// Currently, Values() supports from 1 to $n parameters.
//
$range i 1..n
$for i [[
$range j 1..i
template <$for j, [[typename T$j]]>
internal::ValueArray$i<$for j, [[T$j]]> Values($for j, [[T$j v$j]]) {
return internal::ValueArray$i<$for j, [[T$j]]>($for j, [[v$j]]);
}
]]
// Bool() allows generating tests with parameters in a set of (false, true).
//
// Synopsis:
// Bool()
// - returns a generator producing sequences with elements {false, true}.
//
// It is useful when testing code that depends on Boolean flags. Combinations
// of multiple flags can be tested when several Bool()'s are combined using
// Combine() function.
//
// In the following example all tests in the test case FlagDependentTest
// will be instantiated twice with parameters false and true.
//
// class FlagDependentTest : public testing::TestWithParam<bool> {
// virtual void SetUp() {
// external_flag = GetParam();
// }
// }
// INSTANTIATE_TEST_CASE_P(BoolSequence, FlagDependentTest, Bool());
//
inline internal::ParamGenerator<bool> Bool() {
return Values(false, true);
}
# if GTEST_HAS_COMBINE
// Combine() allows the user to combine two or more sequences to produce
// values of a Cartesian product of those sequences' elements.
//
// Synopsis:
// Combine(gen1, gen2, ..., genN)
// - returns a generator producing sequences with elements coming from
// the Cartesian product of elements from the sequences generated by
// gen1, gen2, ..., genN. The sequence elements will have a type of
// tuple<T1, T2, ..., TN> where T1, T2, ..., TN are the types
// of elements from sequences produces by gen1, gen2, ..., genN.
//
// Combine can have up to $maxtuple arguments. This number is currently limited
// by the maximum number of elements in the tuple implementation used by Google
// Test.
//
// Example:
//
// This will instantiate tests in test case AnimalTest each one with
// the parameter values tuple("cat", BLACK), tuple("cat", WHITE),
// tuple("dog", BLACK), and tuple("dog", WHITE):
//
// enum Color { BLACK, GRAY, WHITE };
// class AnimalTest
// : public testing::TestWithParam<tuple<const char*, Color> > {...};
//
// TEST_P(AnimalTest, AnimalLooksNice) {...}
//
// INSTANTIATE_TEST_CASE_P(AnimalVariations, AnimalTest,
// Combine(Values("cat", "dog"),
// Values(BLACK, WHITE)));
//
// This will instantiate tests in FlagDependentTest with all variations of two
// Boolean flags:
//
// class FlagDependentTest
// : public testing::TestWithParam<tuple(bool, bool)> > {
// virtual void SetUp() {
// // Assigns external_flag_1 and external_flag_2 values from the tuple.
// tie(external_flag_1, external_flag_2) = GetParam();
// }
// };
//
// TEST_P(FlagDependentTest, TestFeature1) {
// // Test your code using external_flag_1 and external_flag_2 here.
// }
// INSTANTIATE_TEST_CASE_P(TwoBoolSequence, FlagDependentTest,
// Combine(Bool(), Bool()));
//
$range i 2..maxtuple
$for i [[
$range j 1..i
template <$for j, [[typename Generator$j]]>
internal::CartesianProductHolder$i<$for j, [[Generator$j]]> Combine(
$for j, [[const Generator$j& g$j]]) {
return internal::CartesianProductHolder$i<$for j, [[Generator$j]]>(
$for j, [[g$j]]);
}
]]
# endif // GTEST_HAS_COMBINE
# define TEST_P(test_case_name, test_name) \
class GTEST_TEST_CLASS_NAME_(test_case_name, test_name) \
: public test_case_name { \
public: \
GTEST_TEST_CLASS_NAME_(test_case_name, test_name)() {} \
virtual void TestBody(); \
private: \
static int AddToRegistry() { \
::testing::UnitTest::GetInstance()->parameterized_test_registry(). \
GetTestCasePatternHolder<test_case_name>(\
#test_case_name, __FILE__, __LINE__)->AddTestPattern(\
#test_case_name, \
#test_name, \
new ::testing::internal::TestMetaFactory< \
GTEST_TEST_CLASS_NAME_(test_case_name, test_name)>()); \
return 0; \
} \
static int gtest_registering_dummy_; \
GTEST_DISALLOW_COPY_AND_ASSIGN_(\
GTEST_TEST_CLASS_NAME_(test_case_name, test_name)); \
}; \
int GTEST_TEST_CLASS_NAME_(test_case_name, \
test_name)::gtest_registering_dummy_ = \
GTEST_TEST_CLASS_NAME_(test_case_name, test_name)::AddToRegistry(); \
void GTEST_TEST_CLASS_NAME_(test_case_name, test_name)::TestBody()
# define INSTANTIATE_TEST_CASE_P(prefix, test_case_name, generator) \
::testing::internal::ParamGenerator<test_case_name::ParamType> \
gtest_##prefix##test_case_name##_EvalGenerator_() { return generator; } \
int gtest_##prefix##test_case_name##_dummy_ = \
::testing::UnitTest::GetInstance()->parameterized_test_registry(). \
GetTestCasePatternHolder<test_case_name>(\
#test_case_name, __FILE__, __LINE__)->AddTestCaseInstantiation(\
#prefix, \
&gtest_##prefix##test_case_name##_EvalGenerator_, \
__FILE__, __LINE__)
} // namespace testing
#endif // GTEST_HAS_PARAM_TEST
#endif // GTEST_INCLUDE_GTEST_GTEST_PARAM_TEST_H_

View File

@ -1,796 +0,0 @@
// Copyright 2007, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Author: wan@google.com (Zhanyong Wan)
// Google Test - The Google C++ Testing Framework
//
// This file implements a universal value printer that can print a
// value of any type T:
//
// void ::testing::internal::UniversalPrinter<T>::Print(value, ostream_ptr);
//
// A user can teach this function how to print a class type T by
// defining either operator<<() or PrintTo() in the namespace that
// defines T. More specifically, the FIRST defined function in the
// following list will be used (assuming T is defined in namespace
// foo):
//
// 1. foo::PrintTo(const T&, ostream*)
// 2. operator<<(ostream&, const T&) defined in either foo or the
// global namespace.
//
// If none of the above is defined, it will print the debug string of
// the value if it is a protocol buffer, or print the raw bytes in the
// value otherwise.
//
// To aid debugging: when T is a reference type, the address of the
// value is also printed; when T is a (const) char pointer, both the
// pointer value and the NUL-terminated string it points to are
// printed.
//
// We also provide some convenient wrappers:
//
// // Prints a value to a string. For a (const or not) char
// // pointer, the NUL-terminated string (but not the pointer) is
// // printed.
// std::string ::testing::PrintToString(const T& value);
//
// // Prints a value tersely: for a reference type, the referenced
// // value (but not the address) is printed; for a (const or not) char
// // pointer, the NUL-terminated string (but not the pointer) is
// // printed.
// void ::testing::internal::UniversalTersePrint(const T& value, ostream*);
//
// // Prints value using the type inferred by the compiler. The difference
// // from UniversalTersePrint() is that this function prints both the
// // pointer and the NUL-terminated string for a (const or not) char pointer.
// void ::testing::internal::UniversalPrint(const T& value, ostream*);
//
// // Prints the fields of a tuple tersely to a string vector, one
// // element for each field. Tuple support must be enabled in
// // gtest-port.h.
// std::vector<string> UniversalTersePrintTupleFieldsToStrings(
// const Tuple& value);
//
// Known limitation:
//
// The print primitives print the elements of an STL-style container
// using the compiler-inferred type of *iter where iter is a
// const_iterator of the container. When const_iterator is an input
// iterator but not a forward iterator, this inferred type may not
// match value_type, and the print output may be incorrect. In
// practice, this is rarely a problem as for most containers
// const_iterator is a forward iterator. We'll fix this if there's an
// actual need for it. Note that this fix cannot rely on value_type
// being defined as many user-defined container types don't have
// value_type.
#ifndef GTEST_INCLUDE_GTEST_GTEST_PRINTERS_H_
#define GTEST_INCLUDE_GTEST_GTEST_PRINTERS_H_
#include <ostream> // NOLINT
#include <sstream>
#include <string>
#include <utility>
#include <vector>
#include "gtest/internal/gtest-port.h"
#include "gtest/internal/gtest-internal.h"
namespace testing {
// Definitions in the 'internal' and 'internal2' name spaces are
// subject to change without notice. DO NOT USE THEM IN USER CODE!
namespace internal2 {
// Prints the given number of bytes in the given object to the given
// ostream.
GTEST_API_ void PrintBytesInObjectTo(const unsigned char* obj_bytes,
size_t count,
::std::ostream* os);
// For selecting which printer to use when a given type has neither <<
// nor PrintTo().
enum TypeKind {
kProtobuf, // a protobuf type
kConvertibleToInteger, // a type implicitly convertible to BiggestInt
// (e.g. a named or unnamed enum type)
kOtherType // anything else
};
// TypeWithoutFormatter<T, kTypeKind>::PrintValue(value, os) is called
// by the universal printer to print a value of type T when neither
// operator<< nor PrintTo() is defined for T, where kTypeKind is the
// "kind" of T as defined by enum TypeKind.
template <typename T, TypeKind kTypeKind>
class TypeWithoutFormatter {
public:
// This default version is called when kTypeKind is kOtherType.
static void PrintValue(const T& value, ::std::ostream* os) {
PrintBytesInObjectTo(reinterpret_cast<const unsigned char*>(&value),
sizeof(value), os);
}
};
// We print a protobuf using its ShortDebugString() when the string
// doesn't exceed this many characters; otherwise we print it using
// DebugString() for better readability.
const size_t kProtobufOneLinerMaxLength = 50;
template <typename T>
class TypeWithoutFormatter<T, kProtobuf> {
public:
static void PrintValue(const T& value, ::std::ostream* os) {
const ::testing::internal::string short_str = value.ShortDebugString();
const ::testing::internal::string pretty_str =
short_str.length() <= kProtobufOneLinerMaxLength ?
short_str : ("\n" + value.DebugString());
*os << ("<" + pretty_str + ">");
}
};
template <typename T>
class TypeWithoutFormatter<T, kConvertibleToInteger> {
public:
// Since T has no << operator or PrintTo() but can be implicitly
// converted to BiggestInt, we print it as a BiggestInt.
//
// Most likely T is an enum type (either named or unnamed), in which
// case printing it as an integer is the desired behavior. In case
// T is not an enum, printing it as an integer is the best we can do
// given that it has no user-defined printer.
static void PrintValue(const T& value, ::std::ostream* os) {
const internal::BiggestInt kBigInt = value;
*os << kBigInt;
}
};
// Prints the given value to the given ostream. If the value is a
// protocol message, its debug string is printed; if it's an enum or
// of a type implicitly convertible to BiggestInt, it's printed as an
// integer; otherwise the bytes in the value are printed. This is
// what UniversalPrinter<T>::Print() does when it knows nothing about
// type T and T has neither << operator nor PrintTo().
//
// A user can override this behavior for a class type Foo by defining
// a << operator in the namespace where Foo is defined.
//
// We put this operator in namespace 'internal2' instead of 'internal'
// to simplify the implementation, as much code in 'internal' needs to
// use << in STL, which would conflict with our own << were it defined
// in 'internal'.
//
// Note that this operator<< takes a generic std::basic_ostream<Char,
// CharTraits> type instead of the more restricted std::ostream. If
// we define it to take an std::ostream instead, we'll get an
// "ambiguous overloads" compiler error when trying to print a type
// Foo that supports streaming to std::basic_ostream<Char,
// CharTraits>, as the compiler cannot tell whether
// operator<<(std::ostream&, const T&) or
// operator<<(std::basic_stream<Char, CharTraits>, const Foo&) is more
// specific.
template <typename Char, typename CharTraits, typename T>
::std::basic_ostream<Char, CharTraits>& operator<<(
::std::basic_ostream<Char, CharTraits>& os, const T& x) {
TypeWithoutFormatter<T,
(internal::IsAProtocolMessage<T>::value ? kProtobuf :
internal::ImplicitlyConvertible<const T&, internal::BiggestInt>::value ?
kConvertibleToInteger : kOtherType)>::PrintValue(x, &os);
return os;
}
} // namespace internal2
} // namespace testing
// This namespace MUST NOT BE NESTED IN ::testing, or the name look-up
// magic needed for implementing UniversalPrinter won't work.
namespace testing_internal {
// Used to print a value that is not an STL-style container when the
// user doesn't define PrintTo() for it.
template <typename T>
void DefaultPrintNonContainerTo(const T& value, ::std::ostream* os) {
// With the following statement, during unqualified name lookup,
// testing::internal2::operator<< appears as if it was declared in
// the nearest enclosing namespace that contains both
// ::testing_internal and ::testing::internal2, i.e. the global
// namespace. For more details, refer to the C++ Standard section
// 7.3.4-1 [namespace.udir]. This allows us to fall back onto
// testing::internal2::operator<< in case T doesn't come with a <<
// operator.
//
// We cannot write 'using ::testing::internal2::operator<<;', which
// gcc 3.3 fails to compile due to a compiler bug.
using namespace ::testing::internal2; // NOLINT
// Assuming T is defined in namespace foo, in the next statement,
// the compiler will consider all of:
//
// 1. foo::operator<< (thanks to Koenig look-up),
// 2. ::operator<< (as the current namespace is enclosed in ::),
// 3. testing::internal2::operator<< (thanks to the using statement above).
//
// The operator<< whose type matches T best will be picked.
//
// We deliberately allow #2 to be a candidate, as sometimes it's
// impossible to define #1 (e.g. when foo is ::std, defining
// anything in it is undefined behavior unless you are a compiler
// vendor.).
*os << value;
}
} // namespace testing_internal
namespace testing {
namespace internal {
// UniversalPrinter<T>::Print(value, ostream_ptr) prints the given
// value to the given ostream. The caller must ensure that
// 'ostream_ptr' is not NULL, or the behavior is undefined.
//
// We define UniversalPrinter as a class template (as opposed to a
// function template), as we need to partially specialize it for
// reference types, which cannot be done with function templates.
template <typename T>
class UniversalPrinter;
template <typename T>
void UniversalPrint(const T& value, ::std::ostream* os);
// Used to print an STL-style container when the user doesn't define
// a PrintTo() for it.
template <typename C>
void DefaultPrintTo(IsContainer /* dummy */,
false_type /* is not a pointer */,
const C& container, ::std::ostream* os) {
const size_t kMaxCount = 32; // The maximum number of elements to print.
*os << '{';
size_t count = 0;
for (typename C::const_iterator it = container.begin();
it != container.end(); ++it, ++count) {
if (count > 0) {
*os << ',';
if (count == kMaxCount) { // Enough has been printed.
*os << " ...";
break;
}
}
*os << ' ';
// We cannot call PrintTo(*it, os) here as PrintTo() doesn't
// handle *it being a native array.
internal::UniversalPrint(*it, os);
}
if (count > 0) {
*os << ' ';
}
*os << '}';
}
// Used to print a pointer that is neither a char pointer nor a member
// pointer, when the user doesn't define PrintTo() for it. (A member
// variable pointer or member function pointer doesn't really point to
// a location in the address space. Their representation is
// implementation-defined. Therefore they will be printed as raw
// bytes.)
template <typename T>
void DefaultPrintTo(IsNotContainer /* dummy */,
true_type /* is a pointer */,
T* p, ::std::ostream* os) {
if (p == NULL) {
*os << "NULL";
} else {
// C++ doesn't allow casting from a function pointer to any object
// pointer.
//
// IsTrue() silences warnings: "Condition is always true",
// "unreachable code".
if (IsTrue(ImplicitlyConvertible<T*, const void*>::value)) {
// T is not a function type. We just call << to print p,
// relying on ADL to pick up user-defined << for their pointer
// types, if any.
*os << p;
} else {
// T is a function type, so '*os << p' doesn't do what we want
// (it just prints p as bool). We want to print p as a const
// void*. However, we cannot cast it to const void* directly,
// even using reinterpret_cast, as earlier versions of gcc
// (e.g. 3.4.5) cannot compile the cast when p is a function
// pointer. Casting to UInt64 first solves the problem.
*os << reinterpret_cast<const void*>(
reinterpret_cast<internal::UInt64>(p));
}
}
}
// Used to print a non-container, non-pointer value when the user
// doesn't define PrintTo() for it.
template <typename T>
void DefaultPrintTo(IsNotContainer /* dummy */,
false_type /* is not a pointer */,
const T& value, ::std::ostream* os) {
::testing_internal::DefaultPrintNonContainerTo(value, os);
}
// Prints the given value using the << operator if it has one;
// otherwise prints the bytes in it. This is what
// UniversalPrinter<T>::Print() does when PrintTo() is not specialized
// or overloaded for type T.
//
// A user can override this behavior for a class type Foo by defining
// an overload of PrintTo() in the namespace where Foo is defined. We
// give the user this option as sometimes defining a << operator for
// Foo is not desirable (e.g. the coding style may prevent doing it,
// or there is already a << operator but it doesn't do what the user
// wants).
template <typename T>
void PrintTo(const T& value, ::std::ostream* os) {
// DefaultPrintTo() is overloaded. The type of its first two
// arguments determine which version will be picked. If T is an
// STL-style container, the version for container will be called; if
// T is a pointer, the pointer version will be called; otherwise the
// generic version will be called.
//
// Note that we check for container types here, prior to we check
// for protocol message types in our operator<<. The rationale is:
//
// For protocol messages, we want to give people a chance to
// override Google Mock's format by defining a PrintTo() or
// operator<<. For STL containers, other formats can be
// incompatible with Google Mock's format for the container
// elements; therefore we check for container types here to ensure
// that our format is used.
//
// The second argument of DefaultPrintTo() is needed to bypass a bug
// in Symbian's C++ compiler that prevents it from picking the right
// overload between:
//
// PrintTo(const T& x, ...);
// PrintTo(T* x, ...);
DefaultPrintTo(IsContainerTest<T>(0), is_pointer<T>(), value, os);
}
// The following list of PrintTo() overloads tells
// UniversalPrinter<T>::Print() how to print standard types (built-in
// types, strings, plain arrays, and pointers).
// Overloads for various char types.
GTEST_API_ void PrintTo(unsigned char c, ::std::ostream* os);
GTEST_API_ void PrintTo(signed char c, ::std::ostream* os);
inline void PrintTo(char c, ::std::ostream* os) {
// When printing a plain char, we always treat it as unsigned. This
// way, the output won't be affected by whether the compiler thinks
// char is signed or not.
PrintTo(static_cast<unsigned char>(c), os);
}
// Overloads for other simple built-in types.
inline void PrintTo(bool x, ::std::ostream* os) {
*os << (x ? "true" : "false");
}
// Overload for wchar_t type.
// Prints a wchar_t as a symbol if it is printable or as its internal
// code otherwise and also as its decimal code (except for L'\0').
// The L'\0' char is printed as "L'\\0'". The decimal code is printed
// as signed integer when wchar_t is implemented by the compiler
// as a signed type and is printed as an unsigned integer when wchar_t
// is implemented as an unsigned type.
GTEST_API_ void PrintTo(wchar_t wc, ::std::ostream* os);
// Overloads for C strings.
GTEST_API_ void PrintTo(const char* s, ::std::ostream* os);
inline void PrintTo(char* s, ::std::ostream* os) {
PrintTo(ImplicitCast_<const char*>(s), os);
}
// signed/unsigned char is often used for representing binary data, so
// we print pointers to it as void* to be safe.
inline void PrintTo(const signed char* s, ::std::ostream* os) {
PrintTo(ImplicitCast_<const void*>(s), os);
}
inline void PrintTo(signed char* s, ::std::ostream* os) {
PrintTo(ImplicitCast_<const void*>(s), os);
}
inline void PrintTo(const unsigned char* s, ::std::ostream* os) {
PrintTo(ImplicitCast_<const void*>(s), os);
}
inline void PrintTo(unsigned char* s, ::std::ostream* os) {
PrintTo(ImplicitCast_<const void*>(s), os);
}
// MSVC can be configured to define wchar_t as a typedef of unsigned
// short. It defines _NATIVE_WCHAR_T_DEFINED when wchar_t is a native
// type. When wchar_t is a typedef, defining an overload for const
// wchar_t* would cause unsigned short* be printed as a wide string,
// possibly causing invalid memory accesses.
#if !defined(_MSC_VER) || defined(_NATIVE_WCHAR_T_DEFINED)
// Overloads for wide C strings
GTEST_API_ void PrintTo(const wchar_t* s, ::std::ostream* os);
inline void PrintTo(wchar_t* s, ::std::ostream* os) {
PrintTo(ImplicitCast_<const wchar_t*>(s), os);
}
#endif
// Overload for C arrays. Multi-dimensional arrays are printed
// properly.
// Prints the given number of elements in an array, without printing
// the curly braces.
template <typename T>
void PrintRawArrayTo(const T a[], size_t count, ::std::ostream* os) {
UniversalPrint(a[0], os);
for (size_t i = 1; i != count; i++) {
*os << ", ";
UniversalPrint(a[i], os);
}
}
// Overloads for ::string and ::std::string.
#if GTEST_HAS_GLOBAL_STRING
GTEST_API_ void PrintStringTo(const ::string&s, ::std::ostream* os);
inline void PrintTo(const ::string& s, ::std::ostream* os) {
PrintStringTo(s, os);
}
#endif // GTEST_HAS_GLOBAL_STRING
GTEST_API_ void PrintStringTo(const ::std::string&s, ::std::ostream* os);
inline void PrintTo(const ::std::string& s, ::std::ostream* os) {
PrintStringTo(s, os);
}
// Overloads for ::wstring and ::std::wstring.
#if GTEST_HAS_GLOBAL_WSTRING
GTEST_API_ void PrintWideStringTo(const ::wstring&s, ::std::ostream* os);
inline void PrintTo(const ::wstring& s, ::std::ostream* os) {
PrintWideStringTo(s, os);
}
#endif // GTEST_HAS_GLOBAL_WSTRING
#if GTEST_HAS_STD_WSTRING
GTEST_API_ void PrintWideStringTo(const ::std::wstring&s, ::std::ostream* os);
inline void PrintTo(const ::std::wstring& s, ::std::ostream* os) {
PrintWideStringTo(s, os);
}
#endif // GTEST_HAS_STD_WSTRING
#if GTEST_HAS_TR1_TUPLE
// Overload for ::std::tr1::tuple. Needed for printing function arguments,
// which are packed as tuples.
// Helper function for printing a tuple. T must be instantiated with
// a tuple type.
template <typename T>
void PrintTupleTo(const T& t, ::std::ostream* os);
// Overloaded PrintTo() for tuples of various arities. We support
// tuples of up-to 10 fields. The following implementation works
// regardless of whether tr1::tuple is implemented using the
// non-standard variadic template feature or not.
inline void PrintTo(const ::std::tr1::tuple<>& t, ::std::ostream* os) {
PrintTupleTo(t, os);
}
template <typename T1>
void PrintTo(const ::std::tr1::tuple<T1>& t, ::std::ostream* os) {
PrintTupleTo(t, os);
}
template <typename T1, typename T2>
void PrintTo(const ::std::tr1::tuple<T1, T2>& t, ::std::ostream* os) {
PrintTupleTo(t, os);
}
template <typename T1, typename T2, typename T3>
void PrintTo(const ::std::tr1::tuple<T1, T2, T3>& t, ::std::ostream* os) {
PrintTupleTo(t, os);
}
template <typename T1, typename T2, typename T3, typename T4>
void PrintTo(const ::std::tr1::tuple<T1, T2, T3, T4>& t, ::std::ostream* os) {
PrintTupleTo(t, os);
}
template <typename T1, typename T2, typename T3, typename T4, typename T5>
void PrintTo(const ::std::tr1::tuple<T1, T2, T3, T4, T5>& t,
::std::ostream* os) {
PrintTupleTo(t, os);
}
template <typename T1, typename T2, typename T3, typename T4, typename T5,
typename T6>
void PrintTo(const ::std::tr1::tuple<T1, T2, T3, T4, T5, T6>& t,
::std::ostream* os) {
PrintTupleTo(t, os);
}
template <typename T1, typename T2, typename T3, typename T4, typename T5,
typename T6, typename T7>
void PrintTo(const ::std::tr1::tuple<T1, T2, T3, T4, T5, T6, T7>& t,
::std::ostream* os) {
PrintTupleTo(t, os);
}
template <typename T1, typename T2, typename T3, typename T4, typename T5,
typename T6, typename T7, typename T8>
void PrintTo(const ::std::tr1::tuple<T1, T2, T3, T4, T5, T6, T7, T8>& t,
::std::ostream* os) {
PrintTupleTo(t, os);
}
template <typename T1, typename T2, typename T3, typename T4, typename T5,
typename T6, typename T7, typename T8, typename T9>
void PrintTo(const ::std::tr1::tuple<T1, T2, T3, T4, T5, T6, T7, T8, T9>& t,
::std::ostream* os) {
PrintTupleTo(t, os);
}
template <typename T1, typename T2, typename T3, typename T4, typename T5,
typename T6, typename T7, typename T8, typename T9, typename T10>
void PrintTo(
const ::std::tr1::tuple<T1, T2, T3, T4, T5, T6, T7, T8, T9, T10>& t,
::std::ostream* os) {
PrintTupleTo(t, os);
}
#endif // GTEST_HAS_TR1_TUPLE
// Overload for std::pair.
template <typename T1, typename T2>
void PrintTo(const ::std::pair<T1, T2>& value, ::std::ostream* os) {
*os << '(';
// We cannot use UniversalPrint(value.first, os) here, as T1 may be
// a reference type. The same for printing value.second.
UniversalPrinter<T1>::Print(value.first, os);
*os << ", ";
UniversalPrinter<T2>::Print(value.second, os);
*os << ')';
}
// Implements printing a non-reference type T by letting the compiler
// pick the right overload of PrintTo() for T.
template <typename T>
class UniversalPrinter {
public:
// MSVC warns about adding const to a function type, so we want to
// disable the warning.
#ifdef _MSC_VER
# pragma warning(push) // Saves the current warning state.
# pragma warning(disable:4180) // Temporarily disables warning 4180.
#endif // _MSC_VER
// Note: we deliberately don't call this PrintTo(), as that name
// conflicts with ::testing::internal::PrintTo in the body of the
// function.
static void Print(const T& value, ::std::ostream* os) {
// By default, ::testing::internal::PrintTo() is used for printing
// the value.
//
// Thanks to Koenig look-up, if T is a class and has its own
// PrintTo() function defined in its namespace, that function will
// be visible here. Since it is more specific than the generic ones
// in ::testing::internal, it will be picked by the compiler in the
// following statement - exactly what we want.
PrintTo(value, os);
}
#ifdef _MSC_VER
# pragma warning(pop) // Restores the warning state.
#endif // _MSC_VER
};
// UniversalPrintArray(begin, len, os) prints an array of 'len'
// elements, starting at address 'begin'.
template <typename T>
void UniversalPrintArray(const T* begin, size_t len, ::std::ostream* os) {
if (len == 0) {
*os << "{}";
} else {
*os << "{ ";
const size_t kThreshold = 18;
const size_t kChunkSize = 8;
// If the array has more than kThreshold elements, we'll have to
// omit some details by printing only the first and the last
// kChunkSize elements.
// TODO(wan@google.com): let the user control the threshold using a flag.
if (len <= kThreshold) {
PrintRawArrayTo(begin, len, os);
} else {
PrintRawArrayTo(begin, kChunkSize, os);
*os << ", ..., ";
PrintRawArrayTo(begin + len - kChunkSize, kChunkSize, os);
}
*os << " }";
}
}
// This overload prints a (const) char array compactly.
GTEST_API_ void UniversalPrintArray(const char* begin,
size_t len,
::std::ostream* os);
// Implements printing an array type T[N].
template <typename T, size_t N>
class UniversalPrinter<T[N]> {
public:
// Prints the given array, omitting some elements when there are too
// many.
static void Print(const T (&a)[N], ::std::ostream* os) {
UniversalPrintArray(a, N, os);
}
};
// Implements printing a reference type T&.
template <typename T>
class UniversalPrinter<T&> {
public:
// MSVC warns about adding const to a function type, so we want to
// disable the warning.
#ifdef _MSC_VER
# pragma warning(push) // Saves the current warning state.
# pragma warning(disable:4180) // Temporarily disables warning 4180.
#endif // _MSC_VER
static void Print(const T& value, ::std::ostream* os) {
// Prints the address of the value. We use reinterpret_cast here
// as static_cast doesn't compile when T is a function type.
*os << "@" << reinterpret_cast<const void*>(&value) << " ";
// Then prints the value itself.
UniversalPrint(value, os);
}
#ifdef _MSC_VER
# pragma warning(pop) // Restores the warning state.
#endif // _MSC_VER
};
// Prints a value tersely: for a reference type, the referenced value
// (but not the address) is printed; for a (const) char pointer, the
// NUL-terminated string (but not the pointer) is printed.
template <typename T>
void UniversalTersePrint(const T& value, ::std::ostream* os) {
UniversalPrint(value, os);
}
inline void UniversalTersePrint(const char* str, ::std::ostream* os) {
if (str == NULL) {
*os << "NULL";
} else {
UniversalPrint(string(str), os);
}
}
inline void UniversalTersePrint(char* str, ::std::ostream* os) {
UniversalTersePrint(static_cast<const char*>(str), os);
}
// Prints a value using the type inferred by the compiler. The
// difference between this and UniversalTersePrint() is that for a
// (const) char pointer, this prints both the pointer and the
// NUL-terminated string.
template <typename T>
void UniversalPrint(const T& value, ::std::ostream* os) {
UniversalPrinter<T>::Print(value, os);
}
#if GTEST_HAS_TR1_TUPLE
typedef ::std::vector<string> Strings;
// This helper template allows PrintTo() for tuples and
// UniversalTersePrintTupleFieldsToStrings() to be defined by
// induction on the number of tuple fields. The idea is that
// TuplePrefixPrinter<N>::PrintPrefixTo(t, os) prints the first N
// fields in tuple t, and can be defined in terms of
// TuplePrefixPrinter<N - 1>.
// The inductive case.
template <size_t N>
struct TuplePrefixPrinter {
// Prints the first N fields of a tuple.
template <typename Tuple>
static void PrintPrefixTo(const Tuple& t, ::std::ostream* os) {
TuplePrefixPrinter<N - 1>::PrintPrefixTo(t, os);
*os << ", ";
UniversalPrinter<typename ::std::tr1::tuple_element<N - 1, Tuple>::type>
::Print(::std::tr1::get<N - 1>(t), os);
}
// Tersely prints the first N fields of a tuple to a string vector,
// one element for each field.
template <typename Tuple>
static void TersePrintPrefixToStrings(const Tuple& t, Strings* strings) {
TuplePrefixPrinter<N - 1>::TersePrintPrefixToStrings(t, strings);
::std::stringstream ss;
UniversalTersePrint(::std::tr1::get<N - 1>(t), &ss);
strings->push_back(ss.str());
}
};
// Base cases.
template <>
struct TuplePrefixPrinter<0> {
template <typename Tuple>
static void PrintPrefixTo(const Tuple&, ::std::ostream*) {}
template <typename Tuple>
static void TersePrintPrefixToStrings(const Tuple&, Strings*) {}
};
// We have to specialize the entire TuplePrefixPrinter<> class
// template here, even though the definition of
// TersePrintPrefixToStrings() is the same as the generic version, as
// Embarcadero (formerly CodeGear, formerly Borland) C++ doesn't
// support specializing a method template of a class template.
template <>
struct TuplePrefixPrinter<1> {
template <typename Tuple>
static void PrintPrefixTo(const Tuple& t, ::std::ostream* os) {
UniversalPrinter<typename ::std::tr1::tuple_element<0, Tuple>::type>::
Print(::std::tr1::get<0>(t), os);
}
template <typename Tuple>
static void TersePrintPrefixToStrings(const Tuple& t, Strings* strings) {
::std::stringstream ss;
UniversalTersePrint(::std::tr1::get<0>(t), &ss);
strings->push_back(ss.str());
}
};
// Helper function for printing a tuple. T must be instantiated with
// a tuple type.
template <typename T>
void PrintTupleTo(const T& t, ::std::ostream* os) {
*os << "(";
TuplePrefixPrinter< ::std::tr1::tuple_size<T>::value>::
PrintPrefixTo(t, os);
*os << ")";
}
// Prints the fields of a tuple tersely to a string vector, one
// element for each field. See the comment before
// UniversalTersePrint() for how we define "tersely".
template <typename Tuple>
Strings UniversalTersePrintTupleFieldsToStrings(const Tuple& value) {
Strings result;
TuplePrefixPrinter< ::std::tr1::tuple_size<Tuple>::value>::
TersePrintPrefixToStrings(value, &result);
return result;
}
#endif // GTEST_HAS_TR1_TUPLE
} // namespace internal
template <typename T>
::std::string PrintToString(const T& value) {
::std::stringstream ss;
internal::UniversalTersePrint(value, &ss);
return ss.str();
}
} // namespace testing
#endif // GTEST_INCLUDE_GTEST_GTEST_PRINTERS_H_

View File

@ -1,232 +0,0 @@
// Copyright 2007, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Author: wan@google.com (Zhanyong Wan)
//
// Utilities for testing Google Test itself and code that uses Google Test
// (e.g. frameworks built on top of Google Test).
#ifndef GTEST_INCLUDE_GTEST_GTEST_SPI_H_
#define GTEST_INCLUDE_GTEST_GTEST_SPI_H_
#include "gtest/gtest.h"
namespace testing {
// This helper class can be used to mock out Google Test failure reporting
// so that we can test Google Test or code that builds on Google Test.
//
// An object of this class appends a TestPartResult object to the
// TestPartResultArray object given in the constructor whenever a Google Test
// failure is reported. It can either intercept only failures that are
// generated in the same thread that created this object or it can intercept
// all generated failures. The scope of this mock object can be controlled with
// the second argument to the two arguments constructor.
class GTEST_API_ ScopedFakeTestPartResultReporter
: public TestPartResultReporterInterface {
public:
// The two possible mocking modes of this object.
enum InterceptMode {
INTERCEPT_ONLY_CURRENT_THREAD, // Intercepts only thread local failures.
INTERCEPT_ALL_THREADS // Intercepts all failures.
};
// The c'tor sets this object as the test part result reporter used
// by Google Test. The 'result' parameter specifies where to report the
// results. This reporter will only catch failures generated in the current
// thread. DEPRECATED
explicit ScopedFakeTestPartResultReporter(TestPartResultArray* result);
// Same as above, but you can choose the interception scope of this object.
ScopedFakeTestPartResultReporter(InterceptMode intercept_mode,
TestPartResultArray* result);
// The d'tor restores the previous test part result reporter.
virtual ~ScopedFakeTestPartResultReporter();
// Appends the TestPartResult object to the TestPartResultArray
// received in the constructor.
//
// This method is from the TestPartResultReporterInterface
// interface.
virtual void ReportTestPartResult(const TestPartResult& result);
private:
void Init();
const InterceptMode intercept_mode_;
TestPartResultReporterInterface* old_reporter_;
TestPartResultArray* const result_;
GTEST_DISALLOW_COPY_AND_ASSIGN_(ScopedFakeTestPartResultReporter);
};
namespace internal {
// A helper class for implementing EXPECT_FATAL_FAILURE() and
// EXPECT_NONFATAL_FAILURE(). Its destructor verifies that the given
// TestPartResultArray contains exactly one failure that has the given
// type and contains the given substring. If that's not the case, a
// non-fatal failure will be generated.
class GTEST_API_ SingleFailureChecker {
public:
// The constructor remembers the arguments.
SingleFailureChecker(const TestPartResultArray* results,
TestPartResult::Type type,
const string& substr);
~SingleFailureChecker();
private:
const TestPartResultArray* const results_;
const TestPartResult::Type type_;
const string substr_;
GTEST_DISALLOW_COPY_AND_ASSIGN_(SingleFailureChecker);
};
} // namespace internal
} // namespace testing
// A set of macros for testing Google Test assertions or code that's expected
// to generate Google Test fatal failures. It verifies that the given
// statement will cause exactly one fatal Google Test failure with 'substr'
// being part of the failure message.
//
// There are two different versions of this macro. EXPECT_FATAL_FAILURE only
// affects and considers failures generated in the current thread and
// EXPECT_FATAL_FAILURE_ON_ALL_THREADS does the same but for all threads.
//
// The verification of the assertion is done correctly even when the statement
// throws an exception or aborts the current function.
//
// Known restrictions:
// - 'statement' cannot reference local non-static variables or
// non-static members of the current object.
// - 'statement' cannot return a value.
// - You cannot stream a failure message to this macro.
//
// Note that even though the implementations of the following two
// macros are much alike, we cannot refactor them to use a common
// helper macro, due to some peculiarity in how the preprocessor
// works. The AcceptsMacroThatExpandsToUnprotectedComma test in
// gtest_unittest.cc will fail to compile if we do that.
#define EXPECT_FATAL_FAILURE(statement, substr) \
do { \
class GTestExpectFatalFailureHelper {\
public:\
static void Execute() { statement; }\
};\
::testing::TestPartResultArray gtest_failures;\
::testing::internal::SingleFailureChecker gtest_checker(\
&gtest_failures, ::testing::TestPartResult::kFatalFailure, (substr));\
{\
::testing::ScopedFakeTestPartResultReporter gtest_reporter(\
::testing::ScopedFakeTestPartResultReporter:: \
INTERCEPT_ONLY_CURRENT_THREAD, &gtest_failures);\
GTestExpectFatalFailureHelper::Execute();\
}\
} while (::testing::internal::AlwaysFalse())
#define EXPECT_FATAL_FAILURE_ON_ALL_THREADS(statement, substr) \
do { \
class GTestExpectFatalFailureHelper {\
public:\
static void Execute() { statement; }\
};\
::testing::TestPartResultArray gtest_failures;\
::testing::internal::SingleFailureChecker gtest_checker(\
&gtest_failures, ::testing::TestPartResult::kFatalFailure, (substr));\
{\
::testing::ScopedFakeTestPartResultReporter gtest_reporter(\
::testing::ScopedFakeTestPartResultReporter:: \
INTERCEPT_ALL_THREADS, &gtest_failures);\
GTestExpectFatalFailureHelper::Execute();\
}\
} while (::testing::internal::AlwaysFalse())
// A macro for testing Google Test assertions or code that's expected to
// generate Google Test non-fatal failures. It asserts that the given
// statement will cause exactly one non-fatal Google Test failure with 'substr'
// being part of the failure message.
//
// There are two different versions of this macro. EXPECT_NONFATAL_FAILURE only
// affects and considers failures generated in the current thread and
// EXPECT_NONFATAL_FAILURE_ON_ALL_THREADS does the same but for all threads.
//
// 'statement' is allowed to reference local variables and members of
// the current object.
//
// The verification of the assertion is done correctly even when the statement
// throws an exception or aborts the current function.
//
// Known restrictions:
// - You cannot stream a failure message to this macro.
//
// Note that even though the implementations of the following two
// macros are much alike, we cannot refactor them to use a common
// helper macro, due to some peculiarity in how the preprocessor
// works. If we do that, the code won't compile when the user gives
// EXPECT_NONFATAL_FAILURE() a statement that contains a macro that
// expands to code containing an unprotected comma. The
// AcceptsMacroThatExpandsToUnprotectedComma test in gtest_unittest.cc
// catches that.
//
// For the same reason, we have to write
// if (::testing::internal::AlwaysTrue()) { statement; }
// instead of
// GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement)
// to avoid an MSVC warning on unreachable code.
#define EXPECT_NONFATAL_FAILURE(statement, substr) \
do {\
::testing::TestPartResultArray gtest_failures;\
::testing::internal::SingleFailureChecker gtest_checker(\
&gtest_failures, ::testing::TestPartResult::kNonFatalFailure, \
(substr));\
{\
::testing::ScopedFakeTestPartResultReporter gtest_reporter(\
::testing::ScopedFakeTestPartResultReporter:: \
INTERCEPT_ONLY_CURRENT_THREAD, &gtest_failures);\
if (::testing::internal::AlwaysTrue()) { statement; }\
}\
} while (::testing::internal::AlwaysFalse())
#define EXPECT_NONFATAL_FAILURE_ON_ALL_THREADS(statement, substr) \
do {\
::testing::TestPartResultArray gtest_failures;\
::testing::internal::SingleFailureChecker gtest_checker(\
&gtest_failures, ::testing::TestPartResult::kNonFatalFailure, \
(substr));\
{\
::testing::ScopedFakeTestPartResultReporter gtest_reporter(\
::testing::ScopedFakeTestPartResultReporter::INTERCEPT_ALL_THREADS,\
&gtest_failures);\
if (::testing::internal::AlwaysTrue()) { statement; }\
}\
} while (::testing::internal::AlwaysFalse())
#endif // GTEST_INCLUDE_GTEST_GTEST_SPI_H_

View File

@ -1,176 +0,0 @@
// Copyright 2008, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Author: mheule@google.com (Markus Heule)
//
#ifndef GTEST_INCLUDE_GTEST_GTEST_TEST_PART_H_
#define GTEST_INCLUDE_GTEST_GTEST_TEST_PART_H_
#include <iosfwd>
#include <vector>
#include "gtest/internal/gtest-internal.h"
#include "gtest/internal/gtest-string.h"
namespace testing {
// A copyable object representing the result of a test part (i.e. an
// assertion or an explicit FAIL(), ADD_FAILURE(), or SUCCESS()).
//
// Don't inherit from TestPartResult as its destructor is not virtual.
class GTEST_API_ TestPartResult {
public:
// The possible outcomes of a test part (i.e. an assertion or an
// explicit SUCCEED(), FAIL(), or ADD_FAILURE()).
enum Type {
kSuccess, // Succeeded.
kNonFatalFailure, // Failed but the test can continue.
kFatalFailure // Failed and the test should be terminated.
};
// C'tor. TestPartResult does NOT have a default constructor.
// Always use this constructor (with parameters) to create a
// TestPartResult object.
TestPartResult(Type a_type,
const char* a_file_name,
int a_line_number,
const char* a_message)
: type_(a_type),
file_name_(a_file_name),
line_number_(a_line_number),
summary_(ExtractSummary(a_message)),
message_(a_message) {
}
// Gets the outcome of the test part.
Type type() const { return type_; }
// Gets the name of the source file where the test part took place, or
// NULL if it's unknown.
const char* file_name() const { return file_name_.c_str(); }
// Gets the line in the source file where the test part took place,
// or -1 if it's unknown.
int line_number() const { return line_number_; }
// Gets the summary of the failure message.
const char* summary() const { return summary_.c_str(); }
// Gets the message associated with the test part.
const char* message() const { return message_.c_str(); }
// Returns true iff the test part passed.
bool passed() const { return type_ == kSuccess; }
// Returns true iff the test part failed.
bool failed() const { return type_ != kSuccess; }
// Returns true iff the test part non-fatally failed.
bool nonfatally_failed() const { return type_ == kNonFatalFailure; }
// Returns true iff the test part fatally failed.
bool fatally_failed() const { return type_ == kFatalFailure; }
private:
Type type_;
// Gets the summary of the failure message by omitting the stack
// trace in it.
static internal::String ExtractSummary(const char* message);
// The name of the source file where the test part took place, or
// NULL if the source file is unknown.
internal::String file_name_;
// The line in the source file where the test part took place, or -1
// if the line number is unknown.
int line_number_;
internal::String summary_; // The test failure summary.
internal::String message_; // The test failure message.
};
// Prints a TestPartResult object.
std::ostream& operator<<(std::ostream& os, const TestPartResult& result);
// An array of TestPartResult objects.
//
// Don't inherit from TestPartResultArray as its destructor is not
// virtual.
class GTEST_API_ TestPartResultArray {
public:
TestPartResultArray() {}
// Appends the given TestPartResult to the array.
void Append(const TestPartResult& result);
// Returns the TestPartResult at the given index (0-based).
const TestPartResult& GetTestPartResult(int index) const;
// Returns the number of TestPartResult objects in the array.
int size() const;
private:
std::vector<TestPartResult> array_;
GTEST_DISALLOW_COPY_AND_ASSIGN_(TestPartResultArray);
};
// This interface knows how to report a test part result.
class TestPartResultReporterInterface {
public:
virtual ~TestPartResultReporterInterface() {}
virtual void ReportTestPartResult(const TestPartResult& result) = 0;
};
namespace internal {
// This helper class is used by {ASSERT|EXPECT}_NO_FATAL_FAILURE to check if a
// statement generates new fatal failures. To do so it registers itself as the
// current test part result reporter. Besides checking if fatal failures were
// reported, it only delegates the reporting to the former result reporter.
// The original result reporter is restored in the destructor.
// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM.
class GTEST_API_ HasNewFatalFailureHelper
: public TestPartResultReporterInterface {
public:
HasNewFatalFailureHelper();
virtual ~HasNewFatalFailureHelper();
virtual void ReportTestPartResult(const TestPartResult& result);
bool has_new_fatal_failure() const { return has_new_fatal_failure_; }
private:
bool has_new_fatal_failure_;
TestPartResultReporterInterface* original_reporter_;
GTEST_DISALLOW_COPY_AND_ASSIGN_(HasNewFatalFailureHelper);
};
} // namespace internal
} // namespace testing
#endif // GTEST_INCLUDE_GTEST_GTEST_TEST_PART_H_

View File

@ -1,259 +0,0 @@
// Copyright 2008 Google Inc.
// All Rights Reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Author: wan@google.com (Zhanyong Wan)
#ifndef GTEST_INCLUDE_GTEST_GTEST_TYPED_TEST_H_
#define GTEST_INCLUDE_GTEST_GTEST_TYPED_TEST_H_
// This header implements typed tests and type-parameterized tests.
// Typed (aka type-driven) tests repeat the same test for types in a
// list. You must know which types you want to test with when writing
// typed tests. Here's how you do it:
#if 0
// First, define a fixture class template. It should be parameterized
// by a type. Remember to derive it from testing::Test.
template <typename T>
class FooTest : public testing::Test {
public:
...
typedef std::list<T> List;
static T shared_;
T value_;
};
// Next, associate a list of types with the test case, which will be
// repeated for each type in the list. The typedef is necessary for
// the macro to parse correctly.
typedef testing::Types<char, int, unsigned int> MyTypes;
TYPED_TEST_CASE(FooTest, MyTypes);
// If the type list contains only one type, you can write that type
// directly without Types<...>:
// TYPED_TEST_CASE(FooTest, int);
// Then, use TYPED_TEST() instead of TEST_F() to define as many typed
// tests for this test case as you want.
TYPED_TEST(FooTest, DoesBlah) {
// Inside a test, refer to TypeParam to get the type parameter.
// Since we are inside a derived class template, C++ requires use to
// visit the members of FooTest via 'this'.
TypeParam n = this->value_;
// To visit static members of the fixture, add the TestFixture::
// prefix.
n += TestFixture::shared_;
// To refer to typedefs in the fixture, add the "typename
// TestFixture::" prefix.
typename TestFixture::List values;
values.push_back(n);
...
}
TYPED_TEST(FooTest, HasPropertyA) { ... }
#endif // 0
// Type-parameterized tests are abstract test patterns parameterized
// by a type. Compared with typed tests, type-parameterized tests
// allow you to define the test pattern without knowing what the type
// parameters are. The defined pattern can be instantiated with
// different types any number of times, in any number of translation
// units.
//
// If you are designing an interface or concept, you can define a
// suite of type-parameterized tests to verify properties that any
// valid implementation of the interface/concept should have. Then,
// each implementation can easily instantiate the test suite to verify
// that it conforms to the requirements, without having to write
// similar tests repeatedly. Here's an example:
#if 0
// First, define a fixture class template. It should be parameterized
// by a type. Remember to derive it from testing::Test.
template <typename T>
class FooTest : public testing::Test {
...
};
// Next, declare that you will define a type-parameterized test case
// (the _P suffix is for "parameterized" or "pattern", whichever you
// prefer):
TYPED_TEST_CASE_P(FooTest);
// Then, use TYPED_TEST_P() to define as many type-parameterized tests
// for this type-parameterized test case as you want.
TYPED_TEST_P(FooTest, DoesBlah) {
// Inside a test, refer to TypeParam to get the type parameter.
TypeParam n = 0;
...
}
TYPED_TEST_P(FooTest, HasPropertyA) { ... }
// Now the tricky part: you need to register all test patterns before
// you can instantiate them. The first argument of the macro is the
// test case name; the rest are the names of the tests in this test
// case.
REGISTER_TYPED_TEST_CASE_P(FooTest,
DoesBlah, HasPropertyA);
// Finally, you are free to instantiate the pattern with the types you
// want. If you put the above code in a header file, you can #include
// it in multiple C++ source files and instantiate it multiple times.
//
// To distinguish different instances of the pattern, the first
// argument to the INSTANTIATE_* macro is a prefix that will be added
// to the actual test case name. Remember to pick unique prefixes for
// different instances.
typedef testing::Types<char, int, unsigned int> MyTypes;
INSTANTIATE_TYPED_TEST_CASE_P(My, FooTest, MyTypes);
// If the type list contains only one type, you can write that type
// directly without Types<...>:
// INSTANTIATE_TYPED_TEST_CASE_P(My, FooTest, int);
#endif // 0
#include "gtest/internal/gtest-port.h"
#include "gtest/internal/gtest-type-util.h"
// Implements typed tests.
#if GTEST_HAS_TYPED_TEST
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// Expands to the name of the typedef for the type parameters of the
// given test case.
# define GTEST_TYPE_PARAMS_(TestCaseName) gtest_type_params_##TestCaseName##_
// The 'Types' template argument below must have spaces around it
// since some compilers may choke on '>>' when passing a template
// instance (e.g. Types<int>)
# define TYPED_TEST_CASE(CaseName, Types) \
typedef ::testing::internal::TypeList< Types >::type \
GTEST_TYPE_PARAMS_(CaseName)
# define TYPED_TEST(CaseName, TestName) \
template <typename gtest_TypeParam_> \
class GTEST_TEST_CLASS_NAME_(CaseName, TestName) \
: public CaseName<gtest_TypeParam_> { \
private: \
typedef CaseName<gtest_TypeParam_> TestFixture; \
typedef gtest_TypeParam_ TypeParam; \
virtual void TestBody(); \
}; \
bool gtest_##CaseName##_##TestName##_registered_ GTEST_ATTRIBUTE_UNUSED_ = \
::testing::internal::TypeParameterizedTest< \
CaseName, \
::testing::internal::TemplateSel< \
GTEST_TEST_CLASS_NAME_(CaseName, TestName)>, \
GTEST_TYPE_PARAMS_(CaseName)>::Register(\
"", #CaseName, #TestName, 0); \
template <typename gtest_TypeParam_> \
void GTEST_TEST_CLASS_NAME_(CaseName, TestName)<gtest_TypeParam_>::TestBody()
#endif // GTEST_HAS_TYPED_TEST
// Implements type-parameterized tests.
#if GTEST_HAS_TYPED_TEST_P
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// Expands to the namespace name that the type-parameterized tests for
// the given type-parameterized test case are defined in. The exact
// name of the namespace is subject to change without notice.
# define GTEST_CASE_NAMESPACE_(TestCaseName) \
gtest_case_##TestCaseName##_
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// Expands to the name of the variable used to remember the names of
// the defined tests in the given test case.
# define GTEST_TYPED_TEST_CASE_P_STATE_(TestCaseName) \
gtest_typed_test_case_p_state_##TestCaseName##_
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE DIRECTLY.
//
// Expands to the name of the variable used to remember the names of
// the registered tests in the given test case.
# define GTEST_REGISTERED_TEST_NAMES_(TestCaseName) \
gtest_registered_test_names_##TestCaseName##_
// The variables defined in the type-parameterized test macros are
// static as typically these macros are used in a .h file that can be
// #included in multiple translation units linked together.
# define TYPED_TEST_CASE_P(CaseName) \
static ::testing::internal::TypedTestCasePState \
GTEST_TYPED_TEST_CASE_P_STATE_(CaseName)
# define TYPED_TEST_P(CaseName, TestName) \
namespace GTEST_CASE_NAMESPACE_(CaseName) { \
template <typename gtest_TypeParam_> \
class TestName : public CaseName<gtest_TypeParam_> { \
private: \
typedef CaseName<gtest_TypeParam_> TestFixture; \
typedef gtest_TypeParam_ TypeParam; \
virtual void TestBody(); \
}; \
static bool gtest_##TestName##_defined_ GTEST_ATTRIBUTE_UNUSED_ = \
GTEST_TYPED_TEST_CASE_P_STATE_(CaseName).AddTestName(\
__FILE__, __LINE__, #CaseName, #TestName); \
} \
template <typename gtest_TypeParam_> \
void GTEST_CASE_NAMESPACE_(CaseName)::TestName<gtest_TypeParam_>::TestBody()
# define REGISTER_TYPED_TEST_CASE_P(CaseName, ...) \
namespace GTEST_CASE_NAMESPACE_(CaseName) { \
typedef ::testing::internal::Templates<__VA_ARGS__>::type gtest_AllTests_; \
} \
static const char* const GTEST_REGISTERED_TEST_NAMES_(CaseName) = \
GTEST_TYPED_TEST_CASE_P_STATE_(CaseName).VerifyRegisteredTestNames(\
__FILE__, __LINE__, #__VA_ARGS__)
// The 'Types' template argument below must have spaces around it
// since some compilers may choke on '>>' when passing a template
// instance (e.g. Types<int>)
# define INSTANTIATE_TYPED_TEST_CASE_P(Prefix, CaseName, Types) \
bool gtest_##Prefix##_##CaseName GTEST_ATTRIBUTE_UNUSED_ = \
::testing::internal::TypeParameterizedTestCase<CaseName, \
GTEST_CASE_NAMESPACE_(CaseName)::gtest_AllTests_, \
::testing::internal::TypeList< Types >::type>::Register(\
#Prefix, #CaseName, GTEST_REGISTERED_TEST_NAMES_(CaseName))
#endif // GTEST_HAS_TYPED_TEST_P
#endif // GTEST_INCLUDE_GTEST_GTEST_TYPED_TEST_H_

File diff suppressed because it is too large Load Diff

View File

@ -1,358 +0,0 @@
// Copyright 2006, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
// This file is AUTOMATICALLY GENERATED on 09/24/2010 by command
// 'gen_gtest_pred_impl.py 5'. DO NOT EDIT BY HAND!
//
// Implements a family of generic predicate assertion macros.
#ifndef GTEST_INCLUDE_GTEST_GTEST_PRED_IMPL_H_
#define GTEST_INCLUDE_GTEST_GTEST_PRED_IMPL_H_
// Makes sure this header is not included before gtest.h.
#ifndef GTEST_INCLUDE_GTEST_GTEST_H_
# error Do not include gtest_pred_impl.h directly. Include gtest.h instead.
#endif // GTEST_INCLUDE_GTEST_GTEST_H_
// This header implements a family of generic predicate assertion
// macros:
//
// ASSERT_PRED_FORMAT1(pred_format, v1)
// ASSERT_PRED_FORMAT2(pred_format, v1, v2)
// ...
//
// where pred_format is a function or functor that takes n (in the
// case of ASSERT_PRED_FORMATn) values and their source expression
// text, and returns a testing::AssertionResult. See the definition
// of ASSERT_EQ in gtest.h for an example.
//
// If you don't care about formatting, you can use the more
// restrictive version:
//
// ASSERT_PRED1(pred, v1)
// ASSERT_PRED2(pred, v1, v2)
// ...
//
// where pred is an n-ary function or functor that returns bool,
// and the values v1, v2, ..., must support the << operator for
// streaming to std::ostream.
//
// We also define the EXPECT_* variations.
//
// For now we only support predicates whose arity is at most 5.
// Please email googletestframework@googlegroups.com if you need
// support for higher arities.
// GTEST_ASSERT_ is the basic statement to which all of the assertions
// in this file reduce. Don't use this in your code.
#define GTEST_ASSERT_(expression, on_failure) \
GTEST_AMBIGUOUS_ELSE_BLOCKER_ \
if (const ::testing::AssertionResult gtest_ar = (expression)) \
; \
else \
on_failure(gtest_ar.failure_message())
// Helper function for implementing {EXPECT|ASSERT}_PRED1. Don't use
// this in your code.
template <typename Pred,
typename T1>
AssertionResult AssertPred1Helper(const char* pred_text,
const char* e1,
Pred pred,
const T1& v1) {
if (pred(v1)) return AssertionSuccess();
return AssertionFailure() << pred_text << "("
<< e1 << ") evaluates to false, where"
<< "\n" << e1 << " evaluates to " << v1;
}
// Internal macro for implementing {EXPECT|ASSERT}_PRED_FORMAT1.
// Don't use this in your code.
#define GTEST_PRED_FORMAT1_(pred_format, v1, on_failure)\
GTEST_ASSERT_(pred_format(#v1, v1),\
on_failure)
// Internal macro for implementing {EXPECT|ASSERT}_PRED1. Don't use
// this in your code.
#define GTEST_PRED1_(pred, v1, on_failure)\
GTEST_ASSERT_(::testing::AssertPred1Helper(#pred, \
#v1, \
pred, \
v1), on_failure)
// Unary predicate assertion macros.
#define EXPECT_PRED_FORMAT1(pred_format, v1) \
GTEST_PRED_FORMAT1_(pred_format, v1, GTEST_NONFATAL_FAILURE_)
#define EXPECT_PRED1(pred, v1) \
GTEST_PRED1_(pred, v1, GTEST_NONFATAL_FAILURE_)
#define ASSERT_PRED_FORMAT1(pred_format, v1) \
GTEST_PRED_FORMAT1_(pred_format, v1, GTEST_FATAL_FAILURE_)
#define ASSERT_PRED1(pred, v1) \
GTEST_PRED1_(pred, v1, GTEST_FATAL_FAILURE_)
// Helper function for implementing {EXPECT|ASSERT}_PRED2. Don't use
// this in your code.
template <typename Pred,
typename T1,
typename T2>
AssertionResult AssertPred2Helper(const char* pred_text,
const char* e1,
const char* e2,
Pred pred,
const T1& v1,
const T2& v2) {
if (pred(v1, v2)) return AssertionSuccess();
return AssertionFailure() << pred_text << "("
<< e1 << ", "
<< e2 << ") evaluates to false, where"
<< "\n" << e1 << " evaluates to " << v1
<< "\n" << e2 << " evaluates to " << v2;
}
// Internal macro for implementing {EXPECT|ASSERT}_PRED_FORMAT2.
// Don't use this in your code.
#define GTEST_PRED_FORMAT2_(pred_format, v1, v2, on_failure)\
GTEST_ASSERT_(pred_format(#v1, #v2, v1, v2),\
on_failure)
// Internal macro for implementing {EXPECT|ASSERT}_PRED2. Don't use
// this in your code.
#define GTEST_PRED2_(pred, v1, v2, on_failure)\
GTEST_ASSERT_(::testing::AssertPred2Helper(#pred, \
#v1, \
#v2, \
pred, \
v1, \
v2), on_failure)
// Binary predicate assertion macros.
#define EXPECT_PRED_FORMAT2(pred_format, v1, v2) \
GTEST_PRED_FORMAT2_(pred_format, v1, v2, GTEST_NONFATAL_FAILURE_)
#define EXPECT_PRED2(pred, v1, v2) \
GTEST_PRED2_(pred, v1, v2, GTEST_NONFATAL_FAILURE_)
#define ASSERT_PRED_FORMAT2(pred_format, v1, v2) \
GTEST_PRED_FORMAT2_(pred_format, v1, v2, GTEST_FATAL_FAILURE_)
#define ASSERT_PRED2(pred, v1, v2) \
GTEST_PRED2_(pred, v1, v2, GTEST_FATAL_FAILURE_)
// Helper function for implementing {EXPECT|ASSERT}_PRED3. Don't use
// this in your code.
template <typename Pred,
typename T1,
typename T2,
typename T3>
AssertionResult AssertPred3Helper(const char* pred_text,
const char* e1,
const char* e2,
const char* e3,
Pred pred,
const T1& v1,
const T2& v2,
const T3& v3) {
if (pred(v1, v2, v3)) return AssertionSuccess();
return AssertionFailure() << pred_text << "("
<< e1 << ", "
<< e2 << ", "
<< e3 << ") evaluates to false, where"
<< "\n" << e1 << " evaluates to " << v1
<< "\n" << e2 << " evaluates to " << v2
<< "\n" << e3 << " evaluates to " << v3;
}
// Internal macro for implementing {EXPECT|ASSERT}_PRED_FORMAT3.
// Don't use this in your code.
#define GTEST_PRED_FORMAT3_(pred_format, v1, v2, v3, on_failure)\
GTEST_ASSERT_(pred_format(#v1, #v2, #v3, v1, v2, v3),\
on_failure)
// Internal macro for implementing {EXPECT|ASSERT}_PRED3. Don't use
// this in your code.
#define GTEST_PRED3_(pred, v1, v2, v3, on_failure)\
GTEST_ASSERT_(::testing::AssertPred3Helper(#pred, \
#v1, \
#v2, \
#v3, \
pred, \
v1, \
v2, \
v3), on_failure)
// Ternary predicate assertion macros.
#define EXPECT_PRED_FORMAT3(pred_format, v1, v2, v3) \
GTEST_PRED_FORMAT3_(pred_format, v1, v2, v3, GTEST_NONFATAL_FAILURE_)
#define EXPECT_PRED3(pred, v1, v2, v3) \
GTEST_PRED3_(pred, v1, v2, v3, GTEST_NONFATAL_FAILURE_)
#define ASSERT_PRED_FORMAT3(pred_format, v1, v2, v3) \
GTEST_PRED_FORMAT3_(pred_format, v1, v2, v3, GTEST_FATAL_FAILURE_)
#define ASSERT_PRED3(pred, v1, v2, v3) \
GTEST_PRED3_(pred, v1, v2, v3, GTEST_FATAL_FAILURE_)
// Helper function for implementing {EXPECT|ASSERT}_PRED4. Don't use
// this in your code.
template <typename Pred,
typename T1,
typename T2,
typename T3,
typename T4>
AssertionResult AssertPred4Helper(const char* pred_text,
const char* e1,
const char* e2,
const char* e3,
const char* e4,
Pred pred,
const T1& v1,
const T2& v2,
const T3& v3,
const T4& v4) {
if (pred(v1, v2, v3, v4)) return AssertionSuccess();
return AssertionFailure() << pred_text << "("
<< e1 << ", "
<< e2 << ", "
<< e3 << ", "
<< e4 << ") evaluates to false, where"
<< "\n" << e1 << " evaluates to " << v1
<< "\n" << e2 << " evaluates to " << v2
<< "\n" << e3 << " evaluates to " << v3
<< "\n" << e4 << " evaluates to " << v4;
}
// Internal macro for implementing {EXPECT|ASSERT}_PRED_FORMAT4.
// Don't use this in your code.
#define GTEST_PRED_FORMAT4_(pred_format, v1, v2, v3, v4, on_failure)\
GTEST_ASSERT_(pred_format(#v1, #v2, #v3, #v4, v1, v2, v3, v4),\
on_failure)
// Internal macro for implementing {EXPECT|ASSERT}_PRED4. Don't use
// this in your code.
#define GTEST_PRED4_(pred, v1, v2, v3, v4, on_failure)\
GTEST_ASSERT_(::testing::AssertPred4Helper(#pred, \
#v1, \
#v2, \
#v3, \
#v4, \
pred, \
v1, \
v2, \
v3, \
v4), on_failure)
// 4-ary predicate assertion macros.
#define EXPECT_PRED_FORMAT4(pred_format, v1, v2, v3, v4) \
GTEST_PRED_FORMAT4_(pred_format, v1, v2, v3, v4, GTEST_NONFATAL_FAILURE_)
#define EXPECT_PRED4(pred, v1, v2, v3, v4) \
GTEST_PRED4_(pred, v1, v2, v3, v4, GTEST_NONFATAL_FAILURE_)
#define ASSERT_PRED_FORMAT4(pred_format, v1, v2, v3, v4) \
GTEST_PRED_FORMAT4_(pred_format, v1, v2, v3, v4, GTEST_FATAL_FAILURE_)
#define ASSERT_PRED4(pred, v1, v2, v3, v4) \
GTEST_PRED4_(pred, v1, v2, v3, v4, GTEST_FATAL_FAILURE_)
// Helper function for implementing {EXPECT|ASSERT}_PRED5. Don't use
// this in your code.
template <typename Pred,
typename T1,
typename T2,
typename T3,
typename T4,
typename T5>
AssertionResult AssertPred5Helper(const char* pred_text,
const char* e1,
const char* e2,
const char* e3,
const char* e4,
const char* e5,
Pred pred,
const T1& v1,
const T2& v2,
const T3& v3,
const T4& v4,
const T5& v5) {
if (pred(v1, v2, v3, v4, v5)) return AssertionSuccess();
return AssertionFailure() << pred_text << "("
<< e1 << ", "
<< e2 << ", "
<< e3 << ", "
<< e4 << ", "
<< e5 << ") evaluates to false, where"
<< "\n" << e1 << " evaluates to " << v1
<< "\n" << e2 << " evaluates to " << v2
<< "\n" << e3 << " evaluates to " << v3
<< "\n" << e4 << " evaluates to " << v4
<< "\n" << e5 << " evaluates to " << v5;
}
// Internal macro for implementing {EXPECT|ASSERT}_PRED_FORMAT5.
// Don't use this in your code.
#define GTEST_PRED_FORMAT5_(pred_format, v1, v2, v3, v4, v5, on_failure)\
GTEST_ASSERT_(pred_format(#v1, #v2, #v3, #v4, #v5, v1, v2, v3, v4, v5),\
on_failure)
// Internal macro for implementing {EXPECT|ASSERT}_PRED5. Don't use
// this in your code.
#define GTEST_PRED5_(pred, v1, v2, v3, v4, v5, on_failure)\
GTEST_ASSERT_(::testing::AssertPred5Helper(#pred, \
#v1, \
#v2, \
#v3, \
#v4, \
#v5, \
pred, \
v1, \
v2, \
v3, \
v4, \
v5), on_failure)
// 5-ary predicate assertion macros.
#define EXPECT_PRED_FORMAT5(pred_format, v1, v2, v3, v4, v5) \
GTEST_PRED_FORMAT5_(pred_format, v1, v2, v3, v4, v5, GTEST_NONFATAL_FAILURE_)
#define EXPECT_PRED5(pred, v1, v2, v3, v4, v5) \
GTEST_PRED5_(pred, v1, v2, v3, v4, v5, GTEST_NONFATAL_FAILURE_)
#define ASSERT_PRED_FORMAT5(pred_format, v1, v2, v3, v4, v5) \
GTEST_PRED_FORMAT5_(pred_format, v1, v2, v3, v4, v5, GTEST_FATAL_FAILURE_)
#define ASSERT_PRED5(pred, v1, v2, v3, v4, v5) \
GTEST_PRED5_(pred, v1, v2, v3, v4, v5, GTEST_FATAL_FAILURE_)
#endif // GTEST_INCLUDE_GTEST_GTEST_PRED_IMPL_H_

View File

@ -1,58 +0,0 @@
// Copyright 2006, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Author: wan@google.com (Zhanyong Wan)
//
// Google C++ Testing Framework definitions useful in production code.
#ifndef GTEST_INCLUDE_GTEST_GTEST_PROD_H_
#define GTEST_INCLUDE_GTEST_GTEST_PROD_H_
// When you need to test the private or protected members of a class,
// use the FRIEND_TEST macro to declare your tests as friends of the
// class. For example:
//
// class MyClass {
// private:
// void MyMethod();
// FRIEND_TEST(MyClassTest, MyMethod);
// };
//
// class MyClassTest : public testing::Test {
// // ...
// };
//
// TEST_F(MyClassTest, MyMethod) {
// // Can call MyClass::MyMethod() here.
// }
#define FRIEND_TEST(test_case_name, test_name)\
friend class test_case_name##_##test_name##_Test
#endif // GTEST_INCLUDE_GTEST_GTEST_PROD_H_

View File

@ -1,308 +0,0 @@
// Copyright 2005, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Authors: wan@google.com (Zhanyong Wan), eefacm@gmail.com (Sean Mcafee)
//
// The Google C++ Testing Framework (Google Test)
//
// This header file defines internal utilities needed for implementing
// death tests. They are subject to change without notice.
#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_DEATH_TEST_INTERNAL_H_
#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_DEATH_TEST_INTERNAL_H_
#include "gtest/internal/gtest-internal.h"
#include <stdio.h>
namespace testing {
namespace internal {
GTEST_DECLARE_string_(internal_run_death_test);
// Names of the flags (needed for parsing Google Test flags).
const char kDeathTestStyleFlag[] = "death_test_style";
const char kDeathTestUseFork[] = "death_test_use_fork";
const char kInternalRunDeathTestFlag[] = "internal_run_death_test";
#if GTEST_HAS_DEATH_TEST
// DeathTest is a class that hides much of the complexity of the
// GTEST_DEATH_TEST_ macro. It is abstract; its static Create method
// returns a concrete class that depends on the prevailing death test
// style, as defined by the --gtest_death_test_style and/or
// --gtest_internal_run_death_test flags.
// In describing the results of death tests, these terms are used with
// the corresponding definitions:
//
// exit status: The integer exit information in the format specified
// by wait(2)
// exit code: The integer code passed to exit(3), _exit(2), or
// returned from main()
class GTEST_API_ DeathTest {
public:
// Create returns false if there was an error determining the
// appropriate action to take for the current death test; for example,
// if the gtest_death_test_style flag is set to an invalid value.
// The LastMessage method will return a more detailed message in that
// case. Otherwise, the DeathTest pointer pointed to by the "test"
// argument is set. If the death test should be skipped, the pointer
// is set to NULL; otherwise, it is set to the address of a new concrete
// DeathTest object that controls the execution of the current test.
static bool Create(const char* statement, const RE* regex,
const char* file, int line, DeathTest** test);
DeathTest();
virtual ~DeathTest() { }
// A helper class that aborts a death test when it's deleted.
class ReturnSentinel {
public:
explicit ReturnSentinel(DeathTest* test) : test_(test) { }
~ReturnSentinel() { test_->Abort(TEST_ENCOUNTERED_RETURN_STATEMENT); }
private:
DeathTest* const test_;
GTEST_DISALLOW_COPY_AND_ASSIGN_(ReturnSentinel);
} GTEST_ATTRIBUTE_UNUSED_;
// An enumeration of possible roles that may be taken when a death
// test is encountered. EXECUTE means that the death test logic should
// be executed immediately. OVERSEE means that the program should prepare
// the appropriate environment for a child process to execute the death
// test, then wait for it to complete.
enum TestRole { OVERSEE_TEST, EXECUTE_TEST };
// An enumeration of the three reasons that a test might be aborted.
enum AbortReason {
TEST_ENCOUNTERED_RETURN_STATEMENT,
TEST_THREW_EXCEPTION,
TEST_DID_NOT_DIE
};
// Assumes one of the above roles.
virtual TestRole AssumeRole() = 0;
// Waits for the death test to finish and returns its status.
virtual int Wait() = 0;
// Returns true if the death test passed; that is, the test process
// exited during the test, its exit status matches a user-supplied
// predicate, and its stderr output matches a user-supplied regular
// expression.
// The user-supplied predicate may be a macro expression rather
// than a function pointer or functor, or else Wait and Passed could
// be combined.
virtual bool Passed(bool exit_status_ok) = 0;
// Signals that the death test did not die as expected.
virtual void Abort(AbortReason reason) = 0;
// Returns a human-readable outcome message regarding the outcome of
// the last death test.
static const char* LastMessage();
static void set_last_death_test_message(const String& message);
private:
// A string containing a description of the outcome of the last death test.
static String last_death_test_message_;
GTEST_DISALLOW_COPY_AND_ASSIGN_(DeathTest);
};
// Factory interface for death tests. May be mocked out for testing.
class DeathTestFactory {
public:
virtual ~DeathTestFactory() { }
virtual bool Create(const char* statement, const RE* regex,
const char* file, int line, DeathTest** test) = 0;
};
// A concrete DeathTestFactory implementation for normal use.
class DefaultDeathTestFactory : public DeathTestFactory {
public:
virtual bool Create(const char* statement, const RE* regex,
const char* file, int line, DeathTest** test);
};
// Returns true if exit_status describes a process that was terminated
// by a signal, or exited normally with a nonzero exit code.
GTEST_API_ bool ExitedUnsuccessfully(int exit_status);
// Traps C++ exceptions escaping statement and reports them as test
// failures. Note that trapping SEH exceptions is not implemented here.
# if GTEST_HAS_EXCEPTIONS
# define GTEST_EXECUTE_DEATH_TEST_STATEMENT_(statement, death_test) \
try { \
GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement); \
} catch (const ::std::exception& gtest_exception) { \
fprintf(\
stderr, \
"\n%s: Caught std::exception-derived exception escaping the " \
"death test statement. Exception message: %s\n", \
::testing::internal::FormatFileLocation(__FILE__, __LINE__).c_str(), \
gtest_exception.what()); \
fflush(stderr); \
death_test->Abort(::testing::internal::DeathTest::TEST_THREW_EXCEPTION); \
} catch (...) { \
death_test->Abort(::testing::internal::DeathTest::TEST_THREW_EXCEPTION); \
}
# else
# define GTEST_EXECUTE_DEATH_TEST_STATEMENT_(statement, death_test) \
GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement)
# endif
// This macro is for implementing ASSERT_DEATH*, EXPECT_DEATH*,
// ASSERT_EXIT*, and EXPECT_EXIT*.
# define GTEST_DEATH_TEST_(statement, predicate, regex, fail) \
GTEST_AMBIGUOUS_ELSE_BLOCKER_ \
if (::testing::internal::AlwaysTrue()) { \
const ::testing::internal::RE& gtest_regex = (regex); \
::testing::internal::DeathTest* gtest_dt; \
if (!::testing::internal::DeathTest::Create(#statement, &gtest_regex, \
__FILE__, __LINE__, &gtest_dt)) { \
goto GTEST_CONCAT_TOKEN_(gtest_label_, __LINE__); \
} \
if (gtest_dt != NULL) { \
::testing::internal::scoped_ptr< ::testing::internal::DeathTest> \
gtest_dt_ptr(gtest_dt); \
switch (gtest_dt->AssumeRole()) { \
case ::testing::internal::DeathTest::OVERSEE_TEST: \
if (!gtest_dt->Passed(predicate(gtest_dt->Wait()))) { \
goto GTEST_CONCAT_TOKEN_(gtest_label_, __LINE__); \
} \
break; \
case ::testing::internal::DeathTest::EXECUTE_TEST: { \
::testing::internal::DeathTest::ReturnSentinel \
gtest_sentinel(gtest_dt); \
GTEST_EXECUTE_DEATH_TEST_STATEMENT_(statement, gtest_dt); \
gtest_dt->Abort(::testing::internal::DeathTest::TEST_DID_NOT_DIE); \
break; \
} \
default: \
break; \
} \
} \
} else \
GTEST_CONCAT_TOKEN_(gtest_label_, __LINE__): \
fail(::testing::internal::DeathTest::LastMessage())
// The symbol "fail" here expands to something into which a message
// can be streamed.
// A class representing the parsed contents of the
// --gtest_internal_run_death_test flag, as it existed when
// RUN_ALL_TESTS was called.
class InternalRunDeathTestFlag {
public:
InternalRunDeathTestFlag(const String& a_file,
int a_line,
int an_index,
int a_write_fd)
: file_(a_file), line_(a_line), index_(an_index),
write_fd_(a_write_fd) {}
~InternalRunDeathTestFlag() {
if (write_fd_ >= 0)
posix::Close(write_fd_);
}
String file() const { return file_; }
int line() const { return line_; }
int index() const { return index_; }
int write_fd() const { return write_fd_; }
private:
String file_;
int line_;
int index_;
int write_fd_;
GTEST_DISALLOW_COPY_AND_ASSIGN_(InternalRunDeathTestFlag);
};
// Returns a newly created InternalRunDeathTestFlag object with fields
// initialized from the GTEST_FLAG(internal_run_death_test) flag if
// the flag is specified; otherwise returns NULL.
InternalRunDeathTestFlag* ParseInternalRunDeathTestFlag();
#else // GTEST_HAS_DEATH_TEST
// This macro is used for implementing macros such as
// EXPECT_DEATH_IF_SUPPORTED and ASSERT_DEATH_IF_SUPPORTED on systems where
// death tests are not supported. Those macros must compile on such systems
// iff EXPECT_DEATH and ASSERT_DEATH compile with the same parameters on
// systems that support death tests. This allows one to write such a macro
// on a system that does not support death tests and be sure that it will
// compile on a death-test supporting system.
//
// Parameters:
// statement - A statement that a macro such as EXPECT_DEATH would test
// for program termination. This macro has to make sure this
// statement is compiled but not executed, to ensure that
// EXPECT_DEATH_IF_SUPPORTED compiles with a certain
// parameter iff EXPECT_DEATH compiles with it.
// regex - A regex that a macro such as EXPECT_DEATH would use to test
// the output of statement. This parameter has to be
// compiled but not evaluated by this macro, to ensure that
// this macro only accepts expressions that a macro such as
// EXPECT_DEATH would accept.
// terminator - Must be an empty statement for EXPECT_DEATH_IF_SUPPORTED
// and a return statement for ASSERT_DEATH_IF_SUPPORTED.
// This ensures that ASSERT_DEATH_IF_SUPPORTED will not
// compile inside functions where ASSERT_DEATH doesn't
// compile.
//
// The branch that has an always false condition is used to ensure that
// statement and regex are compiled (and thus syntactically correct) but
// never executed. The unreachable code macro protects the terminator
// statement from generating an 'unreachable code' warning in case
// statement unconditionally returns or throws. The Message constructor at
// the end allows the syntax of streaming additional messages into the
// macro, for compilational compatibility with EXPECT_DEATH/ASSERT_DEATH.
# define GTEST_UNSUPPORTED_DEATH_TEST_(statement, regex, terminator) \
GTEST_AMBIGUOUS_ELSE_BLOCKER_ \
if (::testing::internal::AlwaysTrue()) { \
GTEST_LOG_(WARNING) \
<< "Death tests are not supported on this platform.\n" \
<< "Statement '" #statement "' cannot be verified."; \
} else if (::testing::internal::AlwaysFalse()) { \
::testing::internal::RE::PartialMatch(".*", (regex)); \
GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement); \
terminator; \
} else \
::testing::Message()
#endif // GTEST_HAS_DEATH_TEST
} // namespace internal
} // namespace testing
#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_DEATH_TEST_INTERNAL_H_

View File

@ -1,210 +0,0 @@
// Copyright 2008, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Author: keith.ray@gmail.com (Keith Ray)
//
// Google Test filepath utilities
//
// This header file declares classes and functions used internally by
// Google Test. They are subject to change without notice.
//
// This file is #included in <gtest/internal/gtest-internal.h>.
// Do not include this header file separately!
#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_FILEPATH_H_
#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_FILEPATH_H_
#include "gtest/internal/gtest-string.h"
namespace testing {
namespace internal {
// FilePath - a class for file and directory pathname manipulation which
// handles platform-specific conventions (like the pathname separator).
// Used for helper functions for naming files in a directory for xml output.
// Except for Set methods, all methods are const or static, which provides an
// "immutable value object" -- useful for peace of mind.
// A FilePath with a value ending in a path separator ("like/this/") represents
// a directory, otherwise it is assumed to represent a file. In either case,
// it may or may not represent an actual file or directory in the file system.
// Names are NOT checked for syntax correctness -- no checking for illegal
// characters, malformed paths, etc.
class GTEST_API_ FilePath {
public:
FilePath() : pathname_("") { }
FilePath(const FilePath& rhs) : pathname_(rhs.pathname_) { }
explicit FilePath(const char* pathname) : pathname_(pathname) {
Normalize();
}
explicit FilePath(const String& pathname) : pathname_(pathname) {
Normalize();
}
FilePath& operator=(const FilePath& rhs) {
Set(rhs);
return *this;
}
void Set(const FilePath& rhs) {
pathname_ = rhs.pathname_;
}
String ToString() const { return pathname_; }
const char* c_str() const { return pathname_.c_str(); }
// Returns the current working directory, or "" if unsuccessful.
static FilePath GetCurrentDir();
// Given directory = "dir", base_name = "test", number = 0,
// extension = "xml", returns "dir/test.xml". If number is greater
// than zero (e.g., 12), returns "dir/test_12.xml".
// On Windows platform, uses \ as the separator rather than /.
static FilePath MakeFileName(const FilePath& directory,
const FilePath& base_name,
int number,
const char* extension);
// Given directory = "dir", relative_path = "test.xml",
// returns "dir/test.xml".
// On Windows, uses \ as the separator rather than /.
static FilePath ConcatPaths(const FilePath& directory,
const FilePath& relative_path);
// Returns a pathname for a file that does not currently exist. The pathname
// will be directory/base_name.extension or
// directory/base_name_<number>.extension if directory/base_name.extension
// already exists. The number will be incremented until a pathname is found
// that does not already exist.
// Examples: 'dir/foo_test.xml' or 'dir/foo_test_1.xml'.
// There could be a race condition if two or more processes are calling this
// function at the same time -- they could both pick the same filename.
static FilePath GenerateUniqueFileName(const FilePath& directory,
const FilePath& base_name,
const char* extension);
// Returns true iff the path is NULL or "".
bool IsEmpty() const { return c_str() == NULL || *c_str() == '\0'; }
// If input name has a trailing separator character, removes it and returns
// the name, otherwise return the name string unmodified.
// On Windows platform, uses \ as the separator, other platforms use /.
FilePath RemoveTrailingPathSeparator() const;
// Returns a copy of the FilePath with the directory part removed.
// Example: FilePath("path/to/file").RemoveDirectoryName() returns
// FilePath("file"). If there is no directory part ("just_a_file"), it returns
// the FilePath unmodified. If there is no file part ("just_a_dir/") it
// returns an empty FilePath ("").
// On Windows platform, '\' is the path separator, otherwise it is '/'.
FilePath RemoveDirectoryName() const;
// RemoveFileName returns the directory path with the filename removed.
// Example: FilePath("path/to/file").RemoveFileName() returns "path/to/".
// If the FilePath is "a_file" or "/a_file", RemoveFileName returns
// FilePath("./") or, on Windows, FilePath(".\\"). If the filepath does
// not have a file, like "just/a/dir/", it returns the FilePath unmodified.
// On Windows platform, '\' is the path separator, otherwise it is '/'.
FilePath RemoveFileName() const;
// Returns a copy of the FilePath with the case-insensitive extension removed.
// Example: FilePath("dir/file.exe").RemoveExtension("EXE") returns
// FilePath("dir/file"). If a case-insensitive extension is not
// found, returns a copy of the original FilePath.
FilePath RemoveExtension(const char* extension) const;
// Creates directories so that path exists. Returns true if successful or if
// the directories already exist; returns false if unable to create
// directories for any reason. Will also return false if the FilePath does
// not represent a directory (that is, it doesn't end with a path separator).
bool CreateDirectoriesRecursively() const;
// Create the directory so that path exists. Returns true if successful or
// if the directory already exists; returns false if unable to create the
// directory for any reason, including if the parent directory does not
// exist. Not named "CreateDirectory" because that's a macro on Windows.
bool CreateFolder() const;
// Returns true if FilePath describes something in the file-system,
// either a file, directory, or whatever, and that something exists.
bool FileOrDirectoryExists() const;
// Returns true if pathname describes a directory in the file-system
// that exists.
bool DirectoryExists() const;
// Returns true if FilePath ends with a path separator, which indicates that
// it is intended to represent a directory. Returns false otherwise.
// This does NOT check that a directory (or file) actually exists.
bool IsDirectory() const;
// Returns true if pathname describes a root directory. (Windows has one
// root directory per disk drive.)
bool IsRootDirectory() const;
// Returns true if pathname describes an absolute path.
bool IsAbsolutePath() const;
private:
// Replaces multiple consecutive separators with a single separator.
// For example, "bar///foo" becomes "bar/foo". Does not eliminate other
// redundancies that might be in a pathname involving "." or "..".
//
// A pathname with multiple consecutive separators may occur either through
// user error or as a result of some scripts or APIs that generate a pathname
// with a trailing separator. On other platforms the same API or script
// may NOT generate a pathname with a trailing "/". Then elsewhere that
// pathname may have another "/" and pathname components added to it,
// without checking for the separator already being there.
// The script language and operating system may allow paths like "foo//bar"
// but some of the functions in FilePath will not handle that correctly. In
// particular, RemoveTrailingPathSeparator() only removes one separator, and
// it is called in CreateDirectoriesRecursively() assuming that it will change
// a pathname from directory syntax (trailing separator) to filename syntax.
//
// On Windows this method also replaces the alternate path separator '/' with
// the primary path separator '\\', so that for example "bar\\/\\foo" becomes
// "bar\\foo".
void Normalize();
// Returns a pointer to the last occurence of a valid path separator in
// the FilePath. On Windows, for example, both '/' and '\' are valid path
// separators. Returns NULL if no path separator was found.
const char* FindLastPathSeparator() const;
String pathname_;
}; // class FilePath
} // namespace internal
} // namespace testing
#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_FILEPATH_H_

File diff suppressed because it is too large Load Diff

View File

@ -1,233 +0,0 @@
// Copyright 2003 Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Authors: Dan Egnor (egnor@google.com)
//
// A "smart" pointer type with reference tracking. Every pointer to a
// particular object is kept on a circular linked list. When the last pointer
// to an object is destroyed or reassigned, the object is deleted.
//
// Used properly, this deletes the object when the last reference goes away.
// There are several caveats:
// - Like all reference counting schemes, cycles lead to leaks.
// - Each smart pointer is actually two pointers (8 bytes instead of 4).
// - Every time a pointer is assigned, the entire list of pointers to that
// object is traversed. This class is therefore NOT SUITABLE when there
// will often be more than two or three pointers to a particular object.
// - References are only tracked as long as linked_ptr<> objects are copied.
// If a linked_ptr<> is converted to a raw pointer and back, BAD THINGS
// will happen (double deletion).
//
// A good use of this class is storing object references in STL containers.
// You can safely put linked_ptr<> in a vector<>.
// Other uses may not be as good.
//
// Note: If you use an incomplete type with linked_ptr<>, the class
// *containing* linked_ptr<> must have a constructor and destructor (even
// if they do nothing!).
//
// Bill Gibbons suggested we use something like this.
//
// Thread Safety:
// Unlike other linked_ptr implementations, in this implementation
// a linked_ptr object is thread-safe in the sense that:
// - it's safe to copy linked_ptr objects concurrently,
// - it's safe to copy *from* a linked_ptr and read its underlying
// raw pointer (e.g. via get()) concurrently, and
// - it's safe to write to two linked_ptrs that point to the same
// shared object concurrently.
// TODO(wan@google.com): rename this to safe_linked_ptr to avoid
// confusion with normal linked_ptr.
#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_LINKED_PTR_H_
#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_LINKED_PTR_H_
#include <stdlib.h>
#include <assert.h>
#include "gtest/internal/gtest-port.h"
namespace testing {
namespace internal {
// Protects copying of all linked_ptr objects.
GTEST_API_ GTEST_DECLARE_STATIC_MUTEX_(g_linked_ptr_mutex);
// This is used internally by all instances of linked_ptr<>. It needs to be
// a non-template class because different types of linked_ptr<> can refer to
// the same object (linked_ptr<Superclass>(obj) vs linked_ptr<Subclass>(obj)).
// So, it needs to be possible for different types of linked_ptr to participate
// in the same circular linked list, so we need a single class type here.
//
// DO NOT USE THIS CLASS DIRECTLY YOURSELF. Use linked_ptr<T>.
class linked_ptr_internal {
public:
// Create a new circle that includes only this instance.
void join_new() {
next_ = this;
}
// Many linked_ptr operations may change p.link_ for some linked_ptr
// variable p in the same circle as this object. Therefore we need
// to prevent two such operations from occurring concurrently.
//
// Note that different types of linked_ptr objects can coexist in a
// circle (e.g. linked_ptr<Base>, linked_ptr<Derived1>, and
// linked_ptr<Derived2>). Therefore we must use a single mutex to
// protect all linked_ptr objects. This can create serious
// contention in production code, but is acceptable in a testing
// framework.
// Join an existing circle.
// L < g_linked_ptr_mutex
void join(linked_ptr_internal const* ptr) {
MutexLock lock(&g_linked_ptr_mutex);
linked_ptr_internal const* p = ptr;
while (p->next_ != ptr) p = p->next_;
p->next_ = this;
next_ = ptr;
}
// Leave whatever circle we're part of. Returns true if we were the
// last member of the circle. Once this is done, you can join() another.
// L < g_linked_ptr_mutex
bool depart() {
MutexLock lock(&g_linked_ptr_mutex);
if (next_ == this) return true;
linked_ptr_internal const* p = next_;
while (p->next_ != this) p = p->next_;
p->next_ = next_;
return false;
}
private:
mutable linked_ptr_internal const* next_;
};
template <typename T>
class linked_ptr {
public:
typedef T element_type;
// Take over ownership of a raw pointer. This should happen as soon as
// possible after the object is created.
explicit linked_ptr(T* ptr = NULL) { capture(ptr); }
~linked_ptr() { depart(); }
// Copy an existing linked_ptr<>, adding ourselves to the list of references.
template <typename U> linked_ptr(linked_ptr<U> const& ptr) { copy(&ptr); }
linked_ptr(linked_ptr const& ptr) { // NOLINT
assert(&ptr != this);
copy(&ptr);
}
// Assignment releases the old value and acquires the new.
template <typename U> linked_ptr& operator=(linked_ptr<U> const& ptr) {
depart();
copy(&ptr);
return *this;
}
linked_ptr& operator=(linked_ptr const& ptr) {
if (&ptr != this) {
depart();
copy(&ptr);
}
return *this;
}
// Smart pointer members.
void reset(T* ptr = NULL) {
depart();
capture(ptr);
}
T* get() const { return value_; }
T* operator->() const { return value_; }
T& operator*() const { return *value_; }
bool operator==(T* p) const { return value_ == p; }
bool operator!=(T* p) const { return value_ != p; }
template <typename U>
bool operator==(linked_ptr<U> const& ptr) const {
return value_ == ptr.get();
}
template <typename U>
bool operator!=(linked_ptr<U> const& ptr) const {
return value_ != ptr.get();
}
private:
template <typename U>
friend class linked_ptr;
T* value_;
linked_ptr_internal link_;
void depart() {
if (link_.depart()) delete value_;
}
void capture(T* ptr) {
value_ = ptr;
link_.join_new();
}
template <typename U> void copy(linked_ptr<U> const* ptr) {
value_ = ptr->get();
if (value_)
link_.join(&ptr->link_);
else
link_.join_new();
}
};
template<typename T> inline
bool operator==(T* ptr, const linked_ptr<T>& x) {
return ptr == x.get();
}
template<typename T> inline
bool operator!=(T* ptr, const linked_ptr<T>& x) {
return ptr != x.get();
}
// A function to convert T* into linked_ptr<T>
// Doing e.g. make_linked_ptr(new FooBarBaz<type>(arg)) is a shorter notation
// for linked_ptr<FooBarBaz<type> >(new FooBarBaz<type>(arg))
template <typename T>
linked_ptr<T> make_linked_ptr(T* ptr) {
return linked_ptr<T>(ptr);
}
} // namespace internal
} // namespace testing
#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_LINKED_PTR_H_

View File

@ -1,301 +0,0 @@
$$ -*- mode: c++; -*-
$var n = 50 $$ Maximum length of Values arguments we want to support.
$var maxtuple = 10 $$ Maximum number of Combine arguments we want to support.
// Copyright 2008 Google Inc.
// All Rights Reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Author: vladl@google.com (Vlad Losev)
// Type and function utilities for implementing parameterized tests.
// This file is generated by a SCRIPT. DO NOT EDIT BY HAND!
//
// Currently Google Test supports at most $n arguments in Values,
// and at most $maxtuple arguments in Combine. Please contact
// googletestframework@googlegroups.com if you need more.
// Please note that the number of arguments to Combine is limited
// by the maximum arity of the implementation of tr1::tuple which is
// currently set at $maxtuple.
#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_GENERATED_H_
#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_GENERATED_H_
// scripts/fuse_gtest.py depends on gtest's own header being #included
// *unconditionally*. Therefore these #includes cannot be moved
// inside #if GTEST_HAS_PARAM_TEST.
#include "gtest/internal/gtest-param-util.h"
#include "gtest/internal/gtest-port.h"
#if GTEST_HAS_PARAM_TEST
namespace testing {
// Forward declarations of ValuesIn(), which is implemented in
// include/gtest/gtest-param-test.h.
template <typename ForwardIterator>
internal::ParamGenerator<
typename ::testing::internal::IteratorTraits<ForwardIterator>::value_type>
ValuesIn(ForwardIterator begin, ForwardIterator end);
template <typename T, size_t N>
internal::ParamGenerator<T> ValuesIn(const T (&array)[N]);
template <class Container>
internal::ParamGenerator<typename Container::value_type> ValuesIn(
const Container& container);
namespace internal {
// Used in the Values() function to provide polymorphic capabilities.
template <typename T1>
class ValueArray1 {
public:
explicit ValueArray1(T1 v1) : v1_(v1) {}
template <typename T>
operator ParamGenerator<T>() const { return ValuesIn(&v1_, &v1_ + 1); }
private:
// No implementation - assignment is unsupported.
void operator=(const ValueArray1& other);
const T1 v1_;
};
$range i 2..n
$for i [[
$range j 1..i
template <$for j, [[typename T$j]]>
class ValueArray$i {
public:
ValueArray$i($for j, [[T$j v$j]]) : $for j, [[v$(j)_(v$j)]] {}
template <typename T>
operator ParamGenerator<T>() const {
const T array[] = {$for j, [[v$(j)_]]};
return ValuesIn(array);
}
private:
// No implementation - assignment is unsupported.
void operator=(const ValueArray$i& other);
$for j [[
const T$j v$(j)_;
]]
};
]]
# if GTEST_HAS_COMBINE
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// Generates values from the Cartesian product of values produced
// by the argument generators.
//
$range i 2..maxtuple
$for i [[
$range j 1..i
$range k 2..i
template <$for j, [[typename T$j]]>
class CartesianProductGenerator$i
: public ParamGeneratorInterface< ::std::tr1::tuple<$for j, [[T$j]]> > {
public:
typedef ::std::tr1::tuple<$for j, [[T$j]]> ParamType;
CartesianProductGenerator$i($for j, [[const ParamGenerator<T$j>& g$j]])
: $for j, [[g$(j)_(g$j)]] {}
virtual ~CartesianProductGenerator$i() {}
virtual ParamIteratorInterface<ParamType>* Begin() const {
return new Iterator(this, $for j, [[g$(j)_, g$(j)_.begin()]]);
}
virtual ParamIteratorInterface<ParamType>* End() const {
return new Iterator(this, $for j, [[g$(j)_, g$(j)_.end()]]);
}
private:
class Iterator : public ParamIteratorInterface<ParamType> {
public:
Iterator(const ParamGeneratorInterface<ParamType>* base, $for j, [[
const ParamGenerator<T$j>& g$j,
const typename ParamGenerator<T$j>::iterator& current$(j)]])
: base_(base),
$for j, [[
begin$(j)_(g$j.begin()), end$(j)_(g$j.end()), current$(j)_(current$j)
]] {
ComputeCurrentValue();
}
virtual ~Iterator() {}
virtual const ParamGeneratorInterface<ParamType>* BaseGenerator() const {
return base_;
}
// Advance should not be called on beyond-of-range iterators
// so no component iterators must be beyond end of range, either.
virtual void Advance() {
assert(!AtEnd());
++current$(i)_;
$for k [[
if (current$(i+2-k)_ == end$(i+2-k)_) {
current$(i+2-k)_ = begin$(i+2-k)_;
++current$(i+2-k-1)_;
}
]]
ComputeCurrentValue();
}
virtual ParamIteratorInterface<ParamType>* Clone() const {
return new Iterator(*this);
}
virtual const ParamType* Current() const { return &current_value_; }
virtual bool Equals(const ParamIteratorInterface<ParamType>& other) const {
// Having the same base generator guarantees that the other
// iterator is of the same type and we can downcast.
GTEST_CHECK_(BaseGenerator() == other.BaseGenerator())
<< "The program attempted to compare iterators "
<< "from different generators." << std::endl;
const Iterator* typed_other =
CheckedDowncastToActualType<const Iterator>(&other);
// We must report iterators equal if they both point beyond their
// respective ranges. That can happen in a variety of fashions,
// so we have to consult AtEnd().
return (AtEnd() && typed_other->AtEnd()) ||
($for j && [[
current$(j)_ == typed_other->current$(j)_
]]);
}
private:
Iterator(const Iterator& other)
: base_(other.base_), $for j, [[
begin$(j)_(other.begin$(j)_),
end$(j)_(other.end$(j)_),
current$(j)_(other.current$(j)_)
]] {
ComputeCurrentValue();
}
void ComputeCurrentValue() {
if (!AtEnd())
current_value_ = ParamType($for j, [[*current$(j)_]]);
}
bool AtEnd() const {
// We must report iterator past the end of the range when either of the
// component iterators has reached the end of its range.
return
$for j || [[
current$(j)_ == end$(j)_
]];
}
// No implementation - assignment is unsupported.
void operator=(const Iterator& other);
const ParamGeneratorInterface<ParamType>* const base_;
// begin[i]_ and end[i]_ define the i-th range that Iterator traverses.
// current[i]_ is the actual traversing iterator.
$for j [[
const typename ParamGenerator<T$j>::iterator begin$(j)_;
const typename ParamGenerator<T$j>::iterator end$(j)_;
typename ParamGenerator<T$j>::iterator current$(j)_;
]]
ParamType current_value_;
}; // class CartesianProductGenerator$i::Iterator
// No implementation - assignment is unsupported.
void operator=(const CartesianProductGenerator$i& other);
$for j [[
const ParamGenerator<T$j> g$(j)_;
]]
}; // class CartesianProductGenerator$i
]]
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// Helper classes providing Combine() with polymorphic features. They allow
// casting CartesianProductGeneratorN<T> to ParamGenerator<U> if T is
// convertible to U.
//
$range i 2..maxtuple
$for i [[
$range j 1..i
template <$for j, [[class Generator$j]]>
class CartesianProductHolder$i {
public:
CartesianProductHolder$i($for j, [[const Generator$j& g$j]])
: $for j, [[g$(j)_(g$j)]] {}
template <$for j, [[typename T$j]]>
operator ParamGenerator< ::std::tr1::tuple<$for j, [[T$j]]> >() const {
return ParamGenerator< ::std::tr1::tuple<$for j, [[T$j]]> >(
new CartesianProductGenerator$i<$for j, [[T$j]]>(
$for j,[[
static_cast<ParamGenerator<T$j> >(g$(j)_)
]]));
}
private:
// No implementation - assignment is unsupported.
void operator=(const CartesianProductHolder$i& other);
$for j [[
const Generator$j g$(j)_;
]]
}; // class CartesianProductHolder$i
]]
# endif // GTEST_HAS_COMBINE
} // namespace internal
} // namespace testing
#endif // GTEST_HAS_PARAM_TEST
#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_GENERATED_H_

View File

@ -1,619 +0,0 @@
// Copyright 2008 Google Inc.
// All Rights Reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Author: vladl@google.com (Vlad Losev)
// Type and function utilities for implementing parameterized tests.
#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_H_
#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_H_
#include <iterator>
#include <utility>
#include <vector>
// scripts/fuse_gtest.py depends on gtest's own header being #included
// *unconditionally*. Therefore these #includes cannot be moved
// inside #if GTEST_HAS_PARAM_TEST.
#include "gtest/internal/gtest-internal.h"
#include "gtest/internal/gtest-linked_ptr.h"
#include "gtest/internal/gtest-port.h"
#include "gtest/gtest-printers.h"
#if GTEST_HAS_PARAM_TEST
namespace testing {
namespace internal {
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// Outputs a message explaining invalid registration of different
// fixture class for the same test case. This may happen when
// TEST_P macro is used to define two tests with the same name
// but in different namespaces.
GTEST_API_ void ReportInvalidTestCaseType(const char* test_case_name,
const char* file, int line);
template <typename> class ParamGeneratorInterface;
template <typename> class ParamGenerator;
// Interface for iterating over elements provided by an implementation
// of ParamGeneratorInterface<T>.
template <typename T>
class ParamIteratorInterface {
public:
virtual ~ParamIteratorInterface() {}
// A pointer to the base generator instance.
// Used only for the purposes of iterator comparison
// to make sure that two iterators belong to the same generator.
virtual const ParamGeneratorInterface<T>* BaseGenerator() const = 0;
// Advances iterator to point to the next element
// provided by the generator. The caller is responsible
// for not calling Advance() on an iterator equal to
// BaseGenerator()->End().
virtual void Advance() = 0;
// Clones the iterator object. Used for implementing copy semantics
// of ParamIterator<T>.
virtual ParamIteratorInterface* Clone() const = 0;
// Dereferences the current iterator and provides (read-only) access
// to the pointed value. It is the caller's responsibility not to call
// Current() on an iterator equal to BaseGenerator()->End().
// Used for implementing ParamGenerator<T>::operator*().
virtual const T* Current() const = 0;
// Determines whether the given iterator and other point to the same
// element in the sequence generated by the generator.
// Used for implementing ParamGenerator<T>::operator==().
virtual bool Equals(const ParamIteratorInterface& other) const = 0;
};
// Class iterating over elements provided by an implementation of
// ParamGeneratorInterface<T>. It wraps ParamIteratorInterface<T>
// and implements the const forward iterator concept.
template <typename T>
class ParamIterator {
public:
typedef T value_type;
typedef const T& reference;
typedef ptrdiff_t difference_type;
// ParamIterator assumes ownership of the impl_ pointer.
ParamIterator(const ParamIterator& other) : impl_(other.impl_->Clone()) {}
ParamIterator& operator=(const ParamIterator& other) {
if (this != &other)
impl_.reset(other.impl_->Clone());
return *this;
}
const T& operator*() const { return *impl_->Current(); }
const T* operator->() const { return impl_->Current(); }
// Prefix version of operator++.
ParamIterator& operator++() {
impl_->Advance();
return *this;
}
// Postfix version of operator++.
ParamIterator operator++(int /*unused*/) {
ParamIteratorInterface<T>* clone = impl_->Clone();
impl_->Advance();
return ParamIterator(clone);
}
bool operator==(const ParamIterator& other) const {
return impl_.get() == other.impl_.get() || impl_->Equals(*other.impl_);
}
bool operator!=(const ParamIterator& other) const {
return !(*this == other);
}
private:
friend class ParamGenerator<T>;
explicit ParamIterator(ParamIteratorInterface<T>* impl) : impl_(impl) {}
scoped_ptr<ParamIteratorInterface<T> > impl_;
};
// ParamGeneratorInterface<T> is the binary interface to access generators
// defined in other translation units.
template <typename T>
class ParamGeneratorInterface {
public:
typedef T ParamType;
virtual ~ParamGeneratorInterface() {}
// Generator interface definition
virtual ParamIteratorInterface<T>* Begin() const = 0;
virtual ParamIteratorInterface<T>* End() const = 0;
};
// Wraps ParamGeneratorInterface<T> and provides general generator syntax
// compatible with the STL Container concept.
// This class implements copy initialization semantics and the contained
// ParamGeneratorInterface<T> instance is shared among all copies
// of the original object. This is possible because that instance is immutable.
template<typename T>
class ParamGenerator {
public:
typedef ParamIterator<T> iterator;
explicit ParamGenerator(ParamGeneratorInterface<T>* impl) : impl_(impl) {}
ParamGenerator(const ParamGenerator& other) : impl_(other.impl_) {}
ParamGenerator& operator=(const ParamGenerator& other) {
impl_ = other.impl_;
return *this;
}
iterator begin() const { return iterator(impl_->Begin()); }
iterator end() const { return iterator(impl_->End()); }
private:
linked_ptr<const ParamGeneratorInterface<T> > impl_;
};
// Generates values from a range of two comparable values. Can be used to
// generate sequences of user-defined types that implement operator+() and
// operator<().
// This class is used in the Range() function.
template <typename T, typename IncrementT>
class RangeGenerator : public ParamGeneratorInterface<T> {
public:
RangeGenerator(T begin, T end, IncrementT step)
: begin_(begin), end_(end),
step_(step), end_index_(CalculateEndIndex(begin, end, step)) {}
virtual ~RangeGenerator() {}
virtual ParamIteratorInterface<T>* Begin() const {
return new Iterator(this, begin_, 0, step_);
}
virtual ParamIteratorInterface<T>* End() const {
return new Iterator(this, end_, end_index_, step_);
}
private:
class Iterator : public ParamIteratorInterface<T> {
public:
Iterator(const ParamGeneratorInterface<T>* base, T value, int index,
IncrementT step)
: base_(base), value_(value), index_(index), step_(step) {}
virtual ~Iterator() {}
virtual const ParamGeneratorInterface<T>* BaseGenerator() const {
return base_;
}
virtual void Advance() {
value_ = value_ + step_;
index_++;
}
virtual ParamIteratorInterface<T>* Clone() const {
return new Iterator(*this);
}
virtual const T* Current() const { return &value_; }
virtual bool Equals(const ParamIteratorInterface<T>& other) const {
// Having the same base generator guarantees that the other
// iterator is of the same type and we can downcast.
GTEST_CHECK_(BaseGenerator() == other.BaseGenerator())
<< "The program attempted to compare iterators "
<< "from different generators." << std::endl;
const int other_index =
CheckedDowncastToActualType<const Iterator>(&other)->index_;
return index_ == other_index;
}
private:
Iterator(const Iterator& other)
: ParamIteratorInterface<T>(),
base_(other.base_), value_(other.value_), index_(other.index_),
step_(other.step_) {}
// No implementation - assignment is unsupported.
void operator=(const Iterator& other);
const ParamGeneratorInterface<T>* const base_;
T value_;
int index_;
const IncrementT step_;
}; // class RangeGenerator::Iterator
static int CalculateEndIndex(const T& begin,
const T& end,
const IncrementT& step) {
int end_index = 0;
for (T i = begin; i < end; i = i + step)
end_index++;
return end_index;
}
// No implementation - assignment is unsupported.
void operator=(const RangeGenerator& other);
const T begin_;
const T end_;
const IncrementT step_;
// The index for the end() iterator. All the elements in the generated
// sequence are indexed (0-based) to aid iterator comparison.
const int end_index_;
}; // class RangeGenerator
// Generates values from a pair of STL-style iterators. Used in the
// ValuesIn() function. The elements are copied from the source range
// since the source can be located on the stack, and the generator
// is likely to persist beyond that stack frame.
template <typename T>
class ValuesInIteratorRangeGenerator : public ParamGeneratorInterface<T> {
public:
template <typename ForwardIterator>
ValuesInIteratorRangeGenerator(ForwardIterator begin, ForwardIterator end)
: container_(begin, end) {}
virtual ~ValuesInIteratorRangeGenerator() {}
virtual ParamIteratorInterface<T>* Begin() const {
return new Iterator(this, container_.begin());
}
virtual ParamIteratorInterface<T>* End() const {
return new Iterator(this, container_.end());
}
private:
typedef typename ::std::vector<T> ContainerType;
class Iterator : public ParamIteratorInterface<T> {
public:
Iterator(const ParamGeneratorInterface<T>* base,
typename ContainerType::const_iterator iterator)
: base_(base), iterator_(iterator) {}
virtual ~Iterator() {}
virtual const ParamGeneratorInterface<T>* BaseGenerator() const {
return base_;
}
virtual void Advance() {
++iterator_;
value_.reset();
}
virtual ParamIteratorInterface<T>* Clone() const {
return new Iterator(*this);
}
// We need to use cached value referenced by iterator_ because *iterator_
// can return a temporary object (and of type other then T), so just
// having "return &*iterator_;" doesn't work.
// value_ is updated here and not in Advance() because Advance()
// can advance iterator_ beyond the end of the range, and we cannot
// detect that fact. The client code, on the other hand, is
// responsible for not calling Current() on an out-of-range iterator.
virtual const T* Current() const {
if (value_.get() == NULL)
value_.reset(new T(*iterator_));
return value_.get();
}
virtual bool Equals(const ParamIteratorInterface<T>& other) const {
// Having the same base generator guarantees that the other
// iterator is of the same type and we can downcast.
GTEST_CHECK_(BaseGenerator() == other.BaseGenerator())
<< "The program attempted to compare iterators "
<< "from different generators." << std::endl;
return iterator_ ==
CheckedDowncastToActualType<const Iterator>(&other)->iterator_;
}
private:
Iterator(const Iterator& other)
// The explicit constructor call suppresses a false warning
// emitted by gcc when supplied with the -Wextra option.
: ParamIteratorInterface<T>(),
base_(other.base_),
iterator_(other.iterator_) {}
const ParamGeneratorInterface<T>* const base_;
typename ContainerType::const_iterator iterator_;
// A cached value of *iterator_. We keep it here to allow access by
// pointer in the wrapping iterator's operator->().
// value_ needs to be mutable to be accessed in Current().
// Use of scoped_ptr helps manage cached value's lifetime,
// which is bound by the lifespan of the iterator itself.
mutable scoped_ptr<const T> value_;
}; // class ValuesInIteratorRangeGenerator::Iterator
// No implementation - assignment is unsupported.
void operator=(const ValuesInIteratorRangeGenerator& other);
const ContainerType container_;
}; // class ValuesInIteratorRangeGenerator
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// Stores a parameter value and later creates tests parameterized with that
// value.
template <class TestClass>
class ParameterizedTestFactory : public TestFactoryBase {
public:
typedef typename TestClass::ParamType ParamType;
explicit ParameterizedTestFactory(ParamType parameter) :
parameter_(parameter) {}
virtual Test* CreateTest() {
TestClass::SetParam(&parameter_);
return new TestClass();
}
private:
const ParamType parameter_;
GTEST_DISALLOW_COPY_AND_ASSIGN_(ParameterizedTestFactory);
};
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// TestMetaFactoryBase is a base class for meta-factories that create
// test factories for passing into MakeAndRegisterTestInfo function.
template <class ParamType>
class TestMetaFactoryBase {
public:
virtual ~TestMetaFactoryBase() {}
virtual TestFactoryBase* CreateTestFactory(ParamType parameter) = 0;
};
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// TestMetaFactory creates test factories for passing into
// MakeAndRegisterTestInfo function. Since MakeAndRegisterTestInfo receives
// ownership of test factory pointer, same factory object cannot be passed
// into that method twice. But ParameterizedTestCaseInfo is going to call
// it for each Test/Parameter value combination. Thus it needs meta factory
// creator class.
template <class TestCase>
class TestMetaFactory
: public TestMetaFactoryBase<typename TestCase::ParamType> {
public:
typedef typename TestCase::ParamType ParamType;
TestMetaFactory() {}
virtual TestFactoryBase* CreateTestFactory(ParamType parameter) {
return new ParameterizedTestFactory<TestCase>(parameter);
}
private:
GTEST_DISALLOW_COPY_AND_ASSIGN_(TestMetaFactory);
};
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// ParameterizedTestCaseInfoBase is a generic interface
// to ParameterizedTestCaseInfo classes. ParameterizedTestCaseInfoBase
// accumulates test information provided by TEST_P macro invocations
// and generators provided by INSTANTIATE_TEST_CASE_P macro invocations
// and uses that information to register all resulting test instances
// in RegisterTests method. The ParameterizeTestCaseRegistry class holds
// a collection of pointers to the ParameterizedTestCaseInfo objects
// and calls RegisterTests() on each of them when asked.
class ParameterizedTestCaseInfoBase {
public:
virtual ~ParameterizedTestCaseInfoBase() {}
// Base part of test case name for display purposes.
virtual const string& GetTestCaseName() const = 0;
// Test case id to verify identity.
virtual TypeId GetTestCaseTypeId() const = 0;
// UnitTest class invokes this method to register tests in this
// test case right before running them in RUN_ALL_TESTS macro.
// This method should not be called more then once on any single
// instance of a ParameterizedTestCaseInfoBase derived class.
virtual void RegisterTests() = 0;
protected:
ParameterizedTestCaseInfoBase() {}
private:
GTEST_DISALLOW_COPY_AND_ASSIGN_(ParameterizedTestCaseInfoBase);
};
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// ParameterizedTestCaseInfo accumulates tests obtained from TEST_P
// macro invocations for a particular test case and generators
// obtained from INSTANTIATE_TEST_CASE_P macro invocations for that
// test case. It registers tests with all values generated by all
// generators when asked.
template <class TestCase>
class ParameterizedTestCaseInfo : public ParameterizedTestCaseInfoBase {
public:
// ParamType and GeneratorCreationFunc are private types but are required
// for declarations of public methods AddTestPattern() and
// AddTestCaseInstantiation().
typedef typename TestCase::ParamType ParamType;
// A function that returns an instance of appropriate generator type.
typedef ParamGenerator<ParamType>(GeneratorCreationFunc)();
explicit ParameterizedTestCaseInfo(const char* name)
: test_case_name_(name) {}
// Test case base name for display purposes.
virtual const string& GetTestCaseName() const { return test_case_name_; }
// Test case id to verify identity.
virtual TypeId GetTestCaseTypeId() const { return GetTypeId<TestCase>(); }
// TEST_P macro uses AddTestPattern() to record information
// about a single test in a LocalTestInfo structure.
// test_case_name is the base name of the test case (without invocation
// prefix). test_base_name is the name of an individual test without
// parameter index. For the test SequenceA/FooTest.DoBar/1 FooTest is
// test case base name and DoBar is test base name.
void AddTestPattern(const char* test_case_name,
const char* test_base_name,
TestMetaFactoryBase<ParamType>* meta_factory) {
tests_.push_back(linked_ptr<TestInfo>(new TestInfo(test_case_name,
test_base_name,
meta_factory)));
}
// INSTANTIATE_TEST_CASE_P macro uses AddGenerator() to record information
// about a generator.
int AddTestCaseInstantiation(const string& instantiation_name,
GeneratorCreationFunc* func,
const char* /* file */,
int /* line */) {
instantiations_.push_back(::std::make_pair(instantiation_name, func));
return 0; // Return value used only to run this method in namespace scope.
}
// UnitTest class invokes this method to register tests in this test case
// test cases right before running tests in RUN_ALL_TESTS macro.
// This method should not be called more then once on any single
// instance of a ParameterizedTestCaseInfoBase derived class.
// UnitTest has a guard to prevent from calling this method more then once.
virtual void RegisterTests() {
for (typename TestInfoContainer::iterator test_it = tests_.begin();
test_it != tests_.end(); ++test_it) {
linked_ptr<TestInfo> test_info = *test_it;
for (typename InstantiationContainer::iterator gen_it =
instantiations_.begin(); gen_it != instantiations_.end();
++gen_it) {
const string& instantiation_name = gen_it->first;
ParamGenerator<ParamType> generator((*gen_it->second)());
Message test_case_name_stream;
if ( !instantiation_name.empty() )
test_case_name_stream << instantiation_name << "/";
test_case_name_stream << test_info->test_case_base_name;
int i = 0;
for (typename ParamGenerator<ParamType>::iterator param_it =
generator.begin();
param_it != generator.end(); ++param_it, ++i) {
Message test_name_stream;
test_name_stream << test_info->test_base_name << "/" << i;
MakeAndRegisterTestInfo(
test_case_name_stream.GetString().c_str(),
test_name_stream.GetString().c_str(),
NULL, // No type parameter.
PrintToString(*param_it).c_str(),
GetTestCaseTypeId(),
TestCase::SetUpTestCase,
TestCase::TearDownTestCase,
test_info->test_meta_factory->CreateTestFactory(*param_it));
} // for param_it
} // for gen_it
} // for test_it
} // RegisterTests
private:
// LocalTestInfo structure keeps information about a single test registered
// with TEST_P macro.
struct TestInfo {
TestInfo(const char* a_test_case_base_name,
const char* a_test_base_name,
TestMetaFactoryBase<ParamType>* a_test_meta_factory) :
test_case_base_name(a_test_case_base_name),
test_base_name(a_test_base_name),
test_meta_factory(a_test_meta_factory) {}
const string test_case_base_name;
const string test_base_name;
const scoped_ptr<TestMetaFactoryBase<ParamType> > test_meta_factory;
};
typedef ::std::vector<linked_ptr<TestInfo> > TestInfoContainer;
// Keeps pairs of <Instantiation name, Sequence generator creation function>
// received from INSTANTIATE_TEST_CASE_P macros.
typedef ::std::vector<std::pair<string, GeneratorCreationFunc*> >
InstantiationContainer;
const string test_case_name_;
TestInfoContainer tests_;
InstantiationContainer instantiations_;
GTEST_DISALLOW_COPY_AND_ASSIGN_(ParameterizedTestCaseInfo);
}; // class ParameterizedTestCaseInfo
// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE.
//
// ParameterizedTestCaseRegistry contains a map of ParameterizedTestCaseInfoBase
// classes accessed by test case names. TEST_P and INSTANTIATE_TEST_CASE_P
// macros use it to locate their corresponding ParameterizedTestCaseInfo
// descriptors.
class ParameterizedTestCaseRegistry {
public:
ParameterizedTestCaseRegistry() {}
~ParameterizedTestCaseRegistry() {
for (TestCaseInfoContainer::iterator it = test_case_infos_.begin();
it != test_case_infos_.end(); ++it) {
delete *it;
}
}
// Looks up or creates and returns a structure containing information about
// tests and instantiations of a particular test case.
template <class TestCase>
ParameterizedTestCaseInfo<TestCase>* GetTestCasePatternHolder(
const char* test_case_name,
const char* file,
int line) {
ParameterizedTestCaseInfo<TestCase>* typed_test_info = NULL;
for (TestCaseInfoContainer::iterator it = test_case_infos_.begin();
it != test_case_infos_.end(); ++it) {
if ((*it)->GetTestCaseName() == test_case_name) {
if ((*it)->GetTestCaseTypeId() != GetTypeId<TestCase>()) {
// Complain about incorrect usage of Google Test facilities
// and terminate the program since we cannot guaranty correct
// test case setup and tear-down in this case.
ReportInvalidTestCaseType(test_case_name, file, line);
posix::Abort();
} else {
// At this point we are sure that the object we found is of the same
// type we are looking for, so we downcast it to that type
// without further checks.
typed_test_info = CheckedDowncastToActualType<
ParameterizedTestCaseInfo<TestCase> >(*it);
}
break;
}
}
if (typed_test_info == NULL) {
typed_test_info = new ParameterizedTestCaseInfo<TestCase>(test_case_name);
test_case_infos_.push_back(typed_test_info);
}
return typed_test_info;
}
void RegisterTests() {
for (TestCaseInfoContainer::iterator it = test_case_infos_.begin();
it != test_case_infos_.end(); ++it) {
(*it)->RegisterTests();
}
}
private:
typedef ::std::vector<ParameterizedTestCaseInfoBase*> TestCaseInfoContainer;
TestCaseInfoContainer test_case_infos_;
GTEST_DISALLOW_COPY_AND_ASSIGN_(ParameterizedTestCaseRegistry);
};
} // namespace internal
} // namespace testing
#endif // GTEST_HAS_PARAM_TEST
#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_H_

File diff suppressed because it is too large Load Diff

View File

@ -1,350 +0,0 @@
// Copyright 2005, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Authors: wan@google.com (Zhanyong Wan), eefacm@gmail.com (Sean Mcafee)
//
// The Google C++ Testing Framework (Google Test)
//
// This header file declares the String class and functions used internally by
// Google Test. They are subject to change without notice. They should not used
// by code external to Google Test.
//
// This header file is #included by <gtest/internal/gtest-internal.h>.
// It should not be #included by other files.
#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_STRING_H_
#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_STRING_H_
#ifdef __BORLANDC__
// string.h is not guaranteed to provide strcpy on C++ Builder.
# include <mem.h>
#endif
#include <string.h>
#include "gtest/internal/gtest-port.h"
#include <string>
namespace testing {
namespace internal {
// String - a UTF-8 string class.
//
// For historic reasons, we don't use std::string.
//
// TODO(wan@google.com): replace this class with std::string or
// implement it in terms of the latter.
//
// Note that String can represent both NULL and the empty string,
// while std::string cannot represent NULL.
//
// NULL and the empty string are considered different. NULL is less
// than anything (including the empty string) except itself.
//
// This class only provides minimum functionality necessary for
// implementing Google Test. We do not intend to implement a full-fledged
// string class here.
//
// Since the purpose of this class is to provide a substitute for
// std::string on platforms where it cannot be used, we define a copy
// constructor and assignment operators such that we don't need
// conditional compilation in a lot of places.
//
// In order to make the representation efficient, the d'tor of String
// is not virtual. Therefore DO NOT INHERIT FROM String.
class GTEST_API_ String {
public:
// Static utility methods
// Returns the input enclosed in double quotes if it's not NULL;
// otherwise returns "(null)". For example, "\"Hello\"" is returned
// for input "Hello".
//
// This is useful for printing a C string in the syntax of a literal.
//
// Known issue: escape sequences are not handled yet.
static String ShowCStringQuoted(const char* c_str);
// Clones a 0-terminated C string, allocating memory using new. The
// caller is responsible for deleting the return value using
// delete[]. Returns the cloned string, or NULL if the input is
// NULL.
//
// This is different from strdup() in string.h, which allocates
// memory using malloc().
static const char* CloneCString(const char* c_str);
#if GTEST_OS_WINDOWS_MOBILE
// Windows CE does not have the 'ANSI' versions of Win32 APIs. To be
// able to pass strings to Win32 APIs on CE we need to convert them
// to 'Unicode', UTF-16.
// Creates a UTF-16 wide string from the given ANSI string, allocating
// memory using new. The caller is responsible for deleting the return
// value using delete[]. Returns the wide string, or NULL if the
// input is NULL.
//
// The wide string is created using the ANSI codepage (CP_ACP) to
// match the behaviour of the ANSI versions of Win32 calls and the
// C runtime.
static LPCWSTR AnsiToUtf16(const char* c_str);
// Creates an ANSI string from the given wide string, allocating
// memory using new. The caller is responsible for deleting the return
// value using delete[]. Returns the ANSI string, or NULL if the
// input is NULL.
//
// The returned string is created using the ANSI codepage (CP_ACP) to
// match the behaviour of the ANSI versions of Win32 calls and the
// C runtime.
static const char* Utf16ToAnsi(LPCWSTR utf16_str);
#endif
// Compares two C strings. Returns true iff they have the same content.
//
// Unlike strcmp(), this function can handle NULL argument(s). A
// NULL C string is considered different to any non-NULL C string,
// including the empty string.
static bool CStringEquals(const char* lhs, const char* rhs);
// Converts a wide C string to a String using the UTF-8 encoding.
// NULL will be converted to "(null)". If an error occurred during
// the conversion, "(failed to convert from wide string)" is
// returned.
static String ShowWideCString(const wchar_t* wide_c_str);
// Similar to ShowWideCString(), except that this function encloses
// the converted string in double quotes.
static String ShowWideCStringQuoted(const wchar_t* wide_c_str);
// Compares two wide C strings. Returns true iff they have the same
// content.
//
// Unlike wcscmp(), this function can handle NULL argument(s). A
// NULL C string is considered different to any non-NULL C string,
// including the empty string.
static bool WideCStringEquals(const wchar_t* lhs, const wchar_t* rhs);
// Compares two C strings, ignoring case. Returns true iff they
// have the same content.
//
// Unlike strcasecmp(), this function can handle NULL argument(s).
// A NULL C string is considered different to any non-NULL C string,
// including the empty string.
static bool CaseInsensitiveCStringEquals(const char* lhs,
const char* rhs);
// Compares two wide C strings, ignoring case. Returns true iff they
// have the same content.
//
// Unlike wcscasecmp(), this function can handle NULL argument(s).
// A NULL C string is considered different to any non-NULL wide C string,
// including the empty string.
// NB: The implementations on different platforms slightly differ.
// On windows, this method uses _wcsicmp which compares according to LC_CTYPE
// environment variable. On GNU platform this method uses wcscasecmp
// which compares according to LC_CTYPE category of the current locale.
// On MacOS X, it uses towlower, which also uses LC_CTYPE category of the
// current locale.
static bool CaseInsensitiveWideCStringEquals(const wchar_t* lhs,
const wchar_t* rhs);
// Formats a list of arguments to a String, using the same format
// spec string as for printf.
//
// We do not use the StringPrintf class as it is not universally
// available.
//
// The result is limited to 4096 characters (including the tailing
// 0). If 4096 characters are not enough to format the input,
// "<buffer exceeded>" is returned.
static String Format(const char* format, ...);
// C'tors
// The default c'tor constructs a NULL string.
String() : c_str_(NULL), length_(0) {}
// Constructs a String by cloning a 0-terminated C string.
String(const char* a_c_str) { // NOLINT
if (a_c_str == NULL) {
c_str_ = NULL;
length_ = 0;
} else {
ConstructNonNull(a_c_str, strlen(a_c_str));
}
}
// Constructs a String by copying a given number of chars from a
// buffer. E.g. String("hello", 3) creates the string "hel",
// String("a\0bcd", 4) creates "a\0bc", String(NULL, 0) creates "",
// and String(NULL, 1) results in access violation.
String(const char* buffer, size_t a_length) {
ConstructNonNull(buffer, a_length);
}
// The copy c'tor creates a new copy of the string. The two
// String objects do not share content.
String(const String& str) : c_str_(NULL), length_(0) { *this = str; }
// D'tor. String is intended to be a final class, so the d'tor
// doesn't need to be virtual.
~String() { delete[] c_str_; }
// Allows a String to be implicitly converted to an ::std::string or
// ::string, and vice versa. Converting a String containing a NULL
// pointer to ::std::string or ::string is undefined behavior.
// Converting a ::std::string or ::string containing an embedded NUL
// character to a String will result in the prefix up to the first
// NUL character.
String(const ::std::string& str) {
ConstructNonNull(str.c_str(), str.length());
}
operator ::std::string() const { return ::std::string(c_str(), length()); }
#if GTEST_HAS_GLOBAL_STRING
String(const ::string& str) {
ConstructNonNull(str.c_str(), str.length());
}
operator ::string() const { return ::string(c_str(), length()); }
#endif // GTEST_HAS_GLOBAL_STRING
// Returns true iff this is an empty string (i.e. "").
bool empty() const { return (c_str() != NULL) && (length() == 0); }
// Compares this with another String.
// Returns < 0 if this is less than rhs, 0 if this is equal to rhs, or > 0
// if this is greater than rhs.
int Compare(const String& rhs) const;
// Returns true iff this String equals the given C string. A NULL
// string and a non-NULL string are considered not equal.
bool operator==(const char* a_c_str) const { return Compare(a_c_str) == 0; }
// Returns true iff this String is less than the given String. A
// NULL string is considered less than "".
bool operator<(const String& rhs) const { return Compare(rhs) < 0; }
// Returns true iff this String doesn't equal the given C string. A NULL
// string and a non-NULL string are considered not equal.
bool operator!=(const char* a_c_str) const { return !(*this == a_c_str); }
// Returns true iff this String ends with the given suffix. *Any*
// String is considered to end with a NULL or empty suffix.
bool EndsWith(const char* suffix) const;
// Returns true iff this String ends with the given suffix, not considering
// case. Any String is considered to end with a NULL or empty suffix.
bool EndsWithCaseInsensitive(const char* suffix) const;
// Returns the length of the encapsulated string, or 0 if the
// string is NULL.
size_t length() const { return length_; }
// Gets the 0-terminated C string this String object represents.
// The String object still owns the string. Therefore the caller
// should NOT delete the return value.
const char* c_str() const { return c_str_; }
// Assigns a C string to this object. Self-assignment works.
const String& operator=(const char* a_c_str) {
return *this = String(a_c_str);
}
// Assigns a String object to this object. Self-assignment works.
const String& operator=(const String& rhs) {
if (this != &rhs) {
delete[] c_str_;
if (rhs.c_str() == NULL) {
c_str_ = NULL;
length_ = 0;
} else {
ConstructNonNull(rhs.c_str(), rhs.length());
}
}
return *this;
}
private:
// Constructs a non-NULL String from the given content. This
// function can only be called when c_str_ has not been allocated.
// ConstructNonNull(NULL, 0) results in an empty string ("").
// ConstructNonNull(NULL, non_zero) is undefined behavior.
void ConstructNonNull(const char* buffer, size_t a_length) {
char* const str = new char[a_length + 1];
memcpy(str, buffer, a_length);
str[a_length] = '\0';
c_str_ = str;
length_ = a_length;
}
const char* c_str_;
size_t length_;
}; // class String
// Streams a String to an ostream. Each '\0' character in the String
// is replaced with "\\0".
inline ::std::ostream& operator<<(::std::ostream& os, const String& str) {
if (str.c_str() == NULL) {
os << "(null)";
} else {
const char* const c_str = str.c_str();
for (size_t i = 0; i != str.length(); i++) {
if (c_str[i] == '\0') {
os << "\\0";
} else {
os << c_str[i];
}
}
}
return os;
}
// Gets the content of the stringstream's buffer as a String. Each '\0'
// character in the buffer is replaced with "\\0".
GTEST_API_ String StringStreamToString(::std::stringstream* stream);
// Converts a streamable value to a String. A NULL pointer is
// converted to "(null)". When the input value is a ::string,
// ::std::string, ::wstring, or ::std::wstring object, each NUL
// character in it is replaced with "\\0".
// Declared here but defined in gtest.h, so that it has access
// to the definition of the Message class, required by the ARM
// compiler.
template <typename T>
String StreamableToString(const T& streamable);
} // namespace internal
} // namespace testing
#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_STRING_H_

Some files were not shown because too many files have changed in this diff Show More