third_codes/PowerJob
1
0
Fork 1
mirror of https://github.com/PowerJob/PowerJob.git synced 2025-07-17 00:00:04 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #5 from PowerJob/master

Browse Source 
Update
This commit is contained in:
Jining Jiang 2021-01-13 14:58:05 +08:00 committed by GitHub
commit 2ab5ac6aa0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
42 changed files with 225 additions and 244 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/FUNDING.yml vendored
View File

@ -2,7 +2,7 @@
github: #[KFCFans] github: #[KFCFans]
patreon: # Replace with a single Patreon username patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username open_collective: powerjob
ko_fi: # Replace with a single Ko-fi username ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry

54
README.md
View File

@ -5,38 +5,37 @@ English | [简体中文](./README_zhCN.md)
</p> </p>
<p align="center"> <p align="center">
<a href="https://github.com/KFCFans/PowerJob/actions"><img src="https://github.com/KFCFans/PowerJob/workflows/Java%20CI%20with%20Maven/badge.svg?branch=master" alt="actions"></a> <a href="https://github.com/PowerJob/PowerJob/actions"><img src="https://github.com/PowerJob/PowerJob/workflows/Java%20CI%20with%20Maven/badge.svg?branch=master" alt="actions"></a>
<a href="https://search.maven.org/search?q=com.github.kfcfans"><img alt="Maven Central" src="https://img.shields.io/maven-central/v/com.github.kfcfans/powerjob-worker"></a> <a href="https://search.maven.org/search?q=com.github.kfcfans"><img alt="Maven Central" src="https://img.shields.io/maven-central/v/com.github.kfcfans/powerjob-worker"></a>
<a href="https://github.com/KFCFans/PowerJob/releases"><img alt="GitHub release (latest SemVer)" src="https://img.shields.io/github/v/release/kfcfans/powerjob?color=%23E59866"></a> <a href="https://github.com/PowerJob/PowerJob/releases"><img alt="GitHub release (latest SemVer)" src="https://img.shields.io/github/v/release/kfcfans/powerjob?color=%23E59866"></a>
<a href="https://github.com/KFCFans/PowerJob/blob/master/LICENSE"><img src="https://img.shields.io/github/license/KFCFans/PowerJob" alt="LICENSE"></a> <a href="https://github.com/PowerJob/PowerJob/blob/master/LICENSE"><img src="https://img.shields.io/github/license/KFCFans/PowerJob" alt="LICENSE"></a>
</p> </p>
- Have you ever wondered how cron jobs could be organized orderly? - Have you ever wondered how cron jobs could be organized orderly?
- Have you ever felt upset when scheduling tasks suddenly terminated without any warning? - Have you ever felt upset about tasks that carry with complex dependencies?
- Have you ever felt helpless when batches of business tasks require handling? - Have you ever felt helpless when scheduling tasks suddenly terminated without any warning?
- Have you ever felt depressed about tasks that carry with complex dependencies? - Have you ever felt depressed when batches of business tasks need to be processed in a distributed manner?
Well, PowerJob is there for you, it is the choice of a new generation. It is a powerful, business-oriented scheduling framework that provides distributed computing ability. Based on Akka architecture, it makes everything with scheduling easier. Just with several steps, PowerJob could be deployed and work for you! Well, PowerJob is there for you, it is the choice of a new generation. It is a powerful, business-oriented scheduling framework that provides distributed computing ability. Based on Akka architecture, it makes everything with scheduling easier. Just with several steps, PowerJob could be deployed and work for you!
# Introduction # Introduction
### Features ### Features
- Simple to use: PowerJob provides a friendly front-end Web that allows developers to visually manage tasks (Create, Read, Update and Delete), monitor tasks, and view logs online. - Simple to use: PowerJob provides a friendly front-end Web that allows developers to visually manage tasks, monitor tasks, and view logs online.
- Complete timing strategy: PowerJob supports four different scheduling strategies, including CRON expression, fixed frequency timing, fixed delay timing as well as the Open API. - Complete timing strategy: PowerJob supports four different scheduling strategies, including CRON expression, fixed frequency timing, fixed delay timing as well as the Open API.
- Various execution modes: PowerJob supports four execution modes: stand-alone, broadcast, Map, and MapReduce. It's worth mentioning the Map and MapReduce modes. With several lines of codes, developers could take full advantage of PowerJob's distributed computing ability. - Various execution modes: PowerJob supports four execution modes: stand-alone, broadcast, Map, and MapReduce. **It's worth mentioning the Map and MapReduce modes. With several lines of codes, developers could take full advantage of PowerJob's distributed computing ability**.
- Complete workflow support. PowerJob supports DAG(Directed acyclic graph) based online task configuration. Developers could arrange tasks on the console, while data could be transferred among tasks on the flow. - Complete workflow support: PowerJob supports DAG(Directed acyclic graph) based online task configuration. Developers could arrange tasks on the console, while data could be transferred among tasks on the flow.
- Extensive executor support: PowerJob supports multiple processors, including Spring Beans, ordinary Java objects, Shell, Python and so on. - Extensive executor support: PowerJob supports multiple processors, including Spring Beans, ordinary Java objects, Shell, Python and so on.
- Simple in dependency: PowerJob aims to be simple in dependency. The only dependency is merely database (MySQL / Oracle / MS SQLServer ...), with MongoDB being the extra dependency for storing large log files online. - Simple in dependency: PowerJob aims to be simple in dependency. The only dependency is merely database (MySQL / Oracle / MS SQLServer ...), with MongoDB being the extra dependency for storing large log files.
- High availability and performance: Unlike traditional job-scheduling frameworks that rely on database locks, PowerJob server is lock-free. PowerJob supports unlimited horizontal expansion. It's easy to achieve high availability and performance by deploying as many PowerJob server instances as you need. - High availability and performance: Unlike traditional job-scheduling frameworks that rely on database locks, PowerJob server is lock-free. PowerJob supports unlimited horizontal expansion. It's easy to achieve high availability and performance by deploying as many PowerJob server instances as you need.
- Quick failover and recovery support: Whenever any task failed, PowerJob server would retry according to the configured strategy. As long as there were enough nodes in the cluster, the failed tasks could execute successfully finally. - Quick failover and recovery support: Whenever any task failed, PowerJob server would retry according to the configured strategy. As long as there were enough nodes in the cluster, the failed tasks could execute successfully finally.
- Convenient to run and maintain: PowerJob supports online logging. Logs generated by the worker would be transferred and displayed on the console instantly, therefore reducing the cost of debugging and improving the efficiency significantly.
### Applicable scenes ### Applicable scenes
- Scenarios with timed tasks: such as full synchronization of data at midnight, generating business reports at desired time. - Scenarios with timed tasks: such as full synchronization of data at midnight, generating business reports at desired time.
- Scenarios that require all machines to run tasks simultaneously: such as log cleanup. - Scenarios that require all machines to run tasks simultaneously: such as log cleanup.
- Scenarios that require distributed processing: For example, a large amount of data requires updating, while the stand-alone execution takes quite a lot of time. The Map/MapReduce mode could be applied in which the workers would join the cluster for PowerJob server to dispatch, to speed up the time-consuming process, therefore improving the computing ability of the whole cluster. - Scenarios that require distributed processing: For example, a large amount of data requires updating, while the stand-alone execution takes quite a lot of time. The Map/MapReduce mode could be applied in which the workers would join the cluster for PowerJob server to dispatch, to speed up the time-consuming process, therefore improving the computing ability of the whole cluster.
- Scenarios with delayed tasks: For instance, disposal of overdue orders. - **Scenarios with delayed tasks**: For instance, disposal of overdue orders.
### Design goals ### Design goals
@ -51,25 +50,25 @@ Application password: 123
### Comparison with similar products ### Comparison with similar products
| | QuartZ | xxl-job | SchedulerX 2.0 | PowerJob | | | QuartZ | PowerJob |
| ---------------------------------- | --------------------------------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | | ---------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------ |
| Timing type | CRON | CRON | CRON, fixed frequency, fixed delay, OpenAPI | **CRON, fixed frequency, fixed delay, OpenAPI** | | Timing type | CRON | **CRON, fixed frequency, fixed delay, OpenAPI** |
| Task type | Built-in Java | Built-in Java, GLUE Java, Shell, Python and other scripts | Built-in Java, external Java (FatJar), Shell, Python and other scripts | **Built-in Java, external Java (container), Shell, Python and other scripts** | | Task type | Built-in Java | **Built-in Java, external Java (JVM Container), Shell, Python and other scripts** |
| Distributed strategy | Unsupported | Static sharding | MapReduce dynamic sharding | **MapReduce dynamic sharding** | | Distributed strategy | Unsupported | **MapReduce dynamic sharding** |
| Online task management | Unsupported | Supported | Supported | **Supported** | | Online task management | Unsupported | **Supported** |
| Online logging | Unsupported | Supported | Unsupported | **Supported** | | Online logging | Unsupported | **Supported** |
| Scheduling methods and performance | Based on database lock, there is a performance bottleneck | Based on database lock, there is a performance bottleneck | Unknown | **Lock-free design, high performance without upper limit** | | Scheduling methods and performance | Based on database lock, there is a performance bottleneck | **Lock-free design, high performance without upper limit** |
| Alarm monitoring | Unsupported | Email | SMS | **Email, WebHook, DingTalk. An interface is provided for customization.** | | Alarm monitoring | Unsupported | **Email, WebHook, DingTalk. An interface is provided for customization.** |
| System dependence | Any relational database (MySQL, Oracle ...) supported by JDBC | MySQL | RMB (Public Beta version for free, hey, helping to promote) | **Any relational database (MySQL, Oracle ...) supported by Spring Data Jpa** | | System dependence | Any relational database (MySQL, Oracle ...) supported by JDBC | **Any relational database (MySQL, Oracle ...) supported by Spring Data Jpa** |
| workflow | Unsupported | Unsupported | Supported | **Supported** | | workflow | Unsupported | **Supported** |
# Document # Document
**[Docs](https://www.yuque.com/powerjob/en/introduce)** **[Docs](https://www.yuque.com/powerjob/en/introduce)**
**[中文文档](https://www.yuque.com/powerjob/product)** **[中文文档](https://www.yuque.com/powerjob/guidence/ztn4i5)**
# User Registration # User Registration
[Click to register as PowerJob user and contribute to PowerJob!](https://github.com/KFCFans/PowerJob/issues/6) [Click to register as PowerJob user and contribute to PowerJob!](https://github.com/PowerJob/PowerJob/issues/6)
ღ( ´・ᴗ・\` )ღ Many thanks to the following registered users. ღ( ´・ᴗ・\` )ღ ღ( ´・ᴗ・\` )ღ Many thanks to the following registered users. ღ( ´・ᴗ・\` )ღ
<p style="text-align: center"> <p style="text-align: center">
<img src="https://raw.githubusercontent.com/KFCFans/PowerJob/master/others/images/user.png" alt="PowerJob User" title="PowerJob User"/> <img src="https://raw.githubusercontent.com/KFCFans/PowerJob/master/others/images/user.png" alt="PowerJob User" title="PowerJob User"/>
@ -77,9 +76,8 @@ Application password: 123
# Others # Others
- Welcome to the Gitter Community: [LINK](https://gitter.im/PowerJob/community)
- PowerJob is permanently open source software(Apache License, Version 2.0), please feel free to try, deploy and put into production! - PowerJob is permanently open source software(Apache License, Version 2.0), please feel free to try, deploy and put into production!
- Author of PowerJob (@KFCFans) has abundant time for maintenance, and is willing to provide technical support if you have needs!
- Welcome to contribute to PowerJob, both Pull Requests and Issues are precious. - Welcome to contribute to PowerJob, both Pull Requests and Issues are precious.
- Please STAR PowerJob if it is valuable. ~ = ￣ω￣ = - Please STAR PowerJob if it is valuable. ~ = ￣ω￣ =
- Do you need any help or want to propose suggestions? Please raise Github issues or contact the Author @KFCFans-> `tengjiqi@gmail.com` directly. - Do you need any help or want to propose suggestions? Please raise Github issues or contact the Author @KFCFans-> `tengjiqi@gmail.com` directly.

8
README_zhCN.md
View File

@ -5,10 +5,10 @@
</p> </p>
<p align="center"> <p align="center">
<a href="https://github.com/KFCFans/PowerJob/actions"><img src="https://github.com/KFCFans/PowerJob/workflows/Java%20CI%20with%20Maven/badge.svg?branch=master" alt="actions"></a> <a href="https://github.com/PowerJob/PowerJob/actions"><img src="https://github.com/PowerJob/PowerJob/workflows/Java%20CI%20with%20Maven/badge.svg?branch=master" alt="actions"></a>
<a href="https://search.maven.org/search?q=com.github.kfcfans"><img alt="Maven Central" src="https://img.shields.io/maven-central/v/com.github.kfcfans/powerjob-worker"></a> <a href="https://search.maven.org/search?q=com.github.kfcfans"><img alt="Maven Central" src="https://img.shields.io/maven-central/v/com.github.kfcfans/powerjob-worker"></a>
<a href="https://github.com/KFCFans/PowerJob/releases"><img alt="GitHub release (latest SemVer)" src="https://img.shields.io/github/v/release/kfcfans/powerjob?color=%23E59866"></a> <a href="https://github.com/PowerJob/PowerJob/releases"><img alt="GitHub release (latest SemVer)" src="https://img.shields.io/github/v/release/kfcfans/powerjob?color=%23E59866"></a>
<a href="https://github.com/KFCFans/PowerJob/blob/master/LICENSE"><img src="https://img.shields.io/github/license/KFCFans/PowerJob" alt="LICENSE"></a> <a href="https://github.com/PowerJob/PowerJob/blob/master/LICENSE"><img src="https://img.shields.io/github/license/KFCFans/PowerJob" alt="LICENSE"></a>
</p> </p>
PowerJob（原OhMyScheduler）是全新一代分布式调度与计算框架，能让您轻松完成作业的调度与繁杂任务的分布式计算。 PowerJob（原OhMyScheduler）是全新一代分布式调度与计算框架，能让您轻松完成作业的调度与繁杂任务的分布式计算。
@ -62,7 +62,7 @@ PowerJob 的设计目标为企业级的分布式任务调度平台，即成为
PS：感谢文档翻译平台[breword](https://www.breword.com/)对本项目英文文档翻译做出的巨大贡献！ PS：感谢文档翻译平台[breword](https://www.breword.com/)对本项目英文文档翻译做出的巨大贡献！
# 接入登记 # 接入登记
[点击进行接入登记，为 PowerJob 的发展贡献自己的力量！](https://github.com/KFCFans/PowerJob/issues/6) [点击进行接入登记，为 PowerJob 的发展贡献自己的力量！](https://github.com/PowerJob/PowerJob/issues/6)
ღ( ´・ᴗ・\` )ღ 感谢以下接入用户的大力支持 ღ( ´・ᴗ・\` )ღ ღ( ´・ᴗ・\` )ღ 感谢以下接入用户的大力支持 ღ( ´・ᴗ・\` )ღ

4
pom.xml
View File

@ -19,8 +19,8 @@
</license> </license>
</licenses> </licenses>
<scm> <scm>
<url>https://github.com/KFCFans/PowerJob</url> <url>https://github.com/PowerJob/PowerJob</url>
<connection>https://github.com/KFCFans/PowerJob.git</connection> <connection>https://github.com/PowerJob/PowerJob.git</connection>
</scm> </scm>
<developers> <developers>

4
powerjob-client/pom.xml
View File

@ -10,13 +10,13 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-client</artifactId> <artifactId>powerjob-client</artifactId>
<version>3.4.2</version> <version>3.4.3</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<properties> <properties>
<junit.version>5.6.1</junit.version> <junit.version>5.6.1</junit.version>
<fastjson.version>1.2.68</fastjson.version> <fastjson.version>1.2.68</fastjson.version>
<powerjob.common.version>3.4.2</powerjob.common.version> <powerjob.common.version>3.4.3</powerjob.common.version>
<mvn.shade.plugin.version>3.2.4</mvn.shade.plugin.version> <mvn.shade.plugin.version>3.2.4</mvn.shade.plugin.version>
</properties> </properties>

195
powerjob-client/src/main/java/com/github/kfcfans/powerjob/client/OhMyClient.java
View File

@ -41,8 +41,9 @@ public class OhMyClient {
/** /**
* Init OhMyClient with domain, appName and password. * Init OhMyClient with domain, appName and password.
* @param domain 比如 www.powerjob-server.com（内网域名，自行完成 DNS & Proxy） * @param domain like powerjob-server.apple-inc.com (Intranet Domain)
* @param appName name of the application * @param appName name of the application
* @param password password of the application
*/ */
public OhMyClient(String domain, String appName, String password) { public OhMyClient(String domain, String appName, String password) {
this(Lists.newArrayList(domain), appName, password); this(Lists.newArrayList(domain), appName, password);
@ -50,9 +51,10 @@ public class OhMyClient {
/** /**
* nit OhMyClient with server address, appName and password. * Init OhMyClient with server address, appName and password.
* @param addressList IP:Port address list * @param addressList IP:Port address list, like 192.168.1.1:7700
* @param appName name of the application * @param appName name of the application
* @param password password of the application
*/ */
public OhMyClient(List<String> addressList, String appName, String password) { public OhMyClient(List<String> addressList, String appName, String password) {
@ -79,7 +81,7 @@ public class OhMyClient {
} }
if (StringUtils.isEmpty(currentAddress)) { if (StringUtils.isEmpty(currentAddress)) {
throw new PowerJobException("no server available"); throw new PowerJobException("no server available for OhMyClient");
} }
log.info("[OhMyClient] {}'s OhMyClient bootstrap successfully, using server: {}", appName, currentAddress); log.info("[OhMyClient] {}'s OhMyClient bootstrap successfully, using server: {}", appName, currentAddress);
} }
@ -101,12 +103,12 @@ public class OhMyClient {
/* ************* Job 区 ************* */ /* ************* Job 区 ************* */
/** /**
* 保存任务（包括创建与修改） * Save one Job
* @param request 任务详细参数 * When an ID exists in SaveJobInfoRequest, it is an update operation. Otherwise, it is a crate operation.
* @return 创建的任务ID * @param request Job meta info
* @throws PowerJobException 异常 * @return jobId
*/ */
public ResultDTO<Long> saveJob(SaveJobInfoRequest request) throws PowerJobException { public ResultDTO<Long> saveJob(SaveJobInfoRequest request) {
request.setAppId(appId); request.setAppId(appId);
MediaType jsonType = MediaType.parse("application/json; charset=utf-8"); MediaType jsonType = MediaType.parse("application/json; charset=utf-8");
@ -116,12 +118,11 @@ public class OhMyClient {
} }
/** /**
* 根据 jobId 查询任务信息 * Query JobInfo by jobId
* @param jobId 任务ID * @param jobId jobId
* @return 任务详细信息 * @return Job meta info
* @throws PowerJobException 异常
*/ */
public ResultDTO<JobInfoDTO> fetchJob(Long jobId) throws PowerJobException { public ResultDTO<JobInfoDTO> fetchJob(Long jobId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("jobId", jobId.toString()) .add("jobId", jobId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -131,12 +132,11 @@ public class OhMyClient {
} }
/** /**
* 禁用某个任务 * Disable one Job by jobId
* @param jobId 任务ID * @param jobId jobId
* @return 标准返回对象 * @return Standard return object
* @throws PowerJobException 异常
*/ */
public ResultDTO<Void> disableJob(Long jobId) throws PowerJobException { public ResultDTO<Void> disableJob(Long jobId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("jobId", jobId.toString()) .add("jobId", jobId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -146,12 +146,11 @@ public class OhMyClient {
} }
/** /**
* 启用某个任务 * Enable one job by jobId
* @param jobId 任务ID * @param jobId jobId
* @return 标准返回对象 * @return Standard return object
* @throws PowerJobException 异常
*/ */
public ResultDTO<Void> enableJob(Long jobId) throws PowerJobException { public ResultDTO<Void> enableJob(Long jobId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("jobId", jobId.toString()) .add("jobId", jobId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -161,12 +160,11 @@ public class OhMyClient {
} }
/** /**
* 删除某个任务 * Delete one job by jobId
* @param jobId 任务ID * @param jobId jobId
* @return 标准返回对象 * @return Standard return object
* @throws PowerJobException 异常
*/ */
public ResultDTO<Void> deleteJob(Long jobId) throws PowerJobException { public ResultDTO<Void> deleteJob(Long jobId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("jobId", jobId.toString()) .add("jobId", jobId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -176,14 +174,13 @@ public class OhMyClient {
} }
/** /**
* 运行某个任务 * Run a job once
* @param jobId 任务ID * @param jobId ID of the job to be run
* @param instanceParams 任务实例的参数 * @param instanceParams Runtime parameters of the job (TaskContext#instanceParams)
* @param delayMS 延迟时间，单位毫秒 * @param delayMS Delay time（Milliseconds）
* @return 任务实例ID（instanceId） * @return instanceId
* @throws PowerJobException 异常
*/ */
public ResultDTO<Long> runJob(Long jobId, String instanceParams, long delayMS) throws PowerJobException { public ResultDTO<Long> runJob(Long jobId, String instanceParams, long delayMS) {
FormBody.Builder builder = new FormBody.Builder() FormBody.Builder builder = new FormBody.Builder()
.add("jobId", jobId.toString()) .add("jobId", jobId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -195,18 +192,17 @@ public class OhMyClient {
String post = postHA(OpenAPIConstant.RUN_JOB, builder.build()); String post = postHA(OpenAPIConstant.RUN_JOB, builder.build());
return JSONObject.parseObject(post, LONG_RESULT_TYPE); return JSONObject.parseObject(post, LONG_RESULT_TYPE);
} }
public ResultDTO<Long> runJob(Long jobId) throws PowerJobException { public ResultDTO<Long> runJob(Long jobId) {
return runJob(jobId, null, 0); return runJob(jobId, null, 0);
} }
/* ************* Instance API list ************* */ /* ************* Instance API list ************* */
/** /**
* 停止应用实例 * Stop one job instance
* @param instanceId 应用实例ID * @param instanceId instanceId
* @return true 停止成功，false 停止失败 * @return Standard return object
* @throws PowerJobException 异常
*/ */
public ResultDTO<Void> stopInstance(Long instanceId) throws PowerJobException { public ResultDTO<Void> stopInstance(Long instanceId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("instanceId", instanceId.toString()) .add("instanceId", instanceId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -216,13 +212,12 @@ public class OhMyClient {
} }
/** /**
* 取消任务实例 * Cancel a job instance that is not yet running
* 接口使用条件：调用接口时间与待取消任务的预计执行时间有一定时间间隔，否则不保证可靠性！ * Notice：There is a time interval between the call interface time and the expected execution time of the job instance to be cancelled, otherwise reliability is not guaranteed
* @param instanceId 任务实例ID * @param instanceId instanceId
* @return true 代表取消成功，false 取消失败 * @return Standard return object
* @throws PowerJobException 异常
*/ */
public ResultDTO<Void> cancelInstance(Long instanceId) throws PowerJobException { public ResultDTO<Void> cancelInstance(Long instanceId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("instanceId", instanceId.toString()) .add("instanceId", instanceId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -232,13 +227,12 @@ public class OhMyClient {
} }
/** /**
* 重试任务实例 * Retry failed job instance
* 只有完成状态（成功、失败、手动停止、被取消）的任务才能被重试，且暂不支持工作流内任务实例的重试 * Notice: Only job instance with completion status (success, failure, manually stopped, cancelled) can be retried, and retries of job instances within workflows are not supported yet.
* @param instanceId 任务实例ID * @param instanceId instanceId
* @return true 代表取消成功，false 取消失败 * @return Standard return object
* @throws PowerJobException 异常
*/ */
public ResultDTO<Void> retryInstance(Long instanceId) throws PowerJobException { public ResultDTO<Void> retryInstance(Long instanceId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("instanceId", instanceId.toString()) .add("instanceId", instanceId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -248,12 +242,11 @@ public class OhMyClient {
} }
/** /**
* 查询任务实例状态 * Query status about a job instance
* @param instanceId 应用实例ID * @param instanceId instanceId
* @return {@link InstanceStatus} 的枚举值 * @return {@link InstanceStatus}
* @throws PowerJobException 异常
*/ */
public ResultDTO<Integer> fetchInstanceStatus(Long instanceId) throws PowerJobException { public ResultDTO<Integer> fetchInstanceStatus(Long instanceId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("instanceId", instanceId.toString()) .add("instanceId", instanceId.toString())
.build(); .build();
@ -262,12 +255,11 @@ public class OhMyClient {
} }
/** /**
* 查询任务实例的信息 * Query detail about a job instance
* @param instanceId 任务实例ID * @param instanceId instanceId
* @return 任务实例信息 * @return instance detail
* @throws PowerJobException 潜在的异常
*/ */
public ResultDTO<InstanceInfoDTO> fetchInstanceInfo(Long instanceId) throws PowerJobException { public ResultDTO<InstanceInfoDTO> fetchInstanceInfo(Long instanceId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("instanceId", instanceId.toString()) .add("instanceId", instanceId.toString())
.build(); .build();
@ -277,12 +269,12 @@ public class OhMyClient {
/* ************* Workflow API list ************* */ /* ************* Workflow API list ************* */
/** /**
* 保存工作流（包括创建和修改） * Save one workflow
* @param request 创建/修改 Workflow 请求 * When an ID exists in SaveWorkflowRequest, it is an update operation. Otherwise, it is a crate operation.
* @return 工作流ID * @param request Workflow meta info
* @throws PowerJobException 异常 * @return workflowId
*/ */
public ResultDTO<Long> saveWorkflow(SaveWorkflowRequest request) throws PowerJobException { public ResultDTO<Long> saveWorkflow(SaveWorkflowRequest request) {
request.setAppId(appId); request.setAppId(appId);
MediaType jsonType = MediaType.parse(OmsConstant.JSON_MEDIA_TYPE); MediaType jsonType = MediaType.parse(OmsConstant.JSON_MEDIA_TYPE);
// 中坑记录：用 FastJSON 序列化会导致 Server 接收时 pEWorkflowDAG 为 null，无语.jpg // 中坑记录：用 FastJSON 序列化会导致 Server 接收时 pEWorkflowDAG 为 null，无语.jpg
@ -292,12 +284,11 @@ public class OhMyClient {
} }
/** /**
* 根据 workflowId 查询工作流信息 * Query Workflow by workflowId
* @param workflowId workflowId * @param workflowId workflowId
* @return 工作流信息 * @return Workflow meta info
* @throws PowerJobException 异常
*/ */
public ResultDTO<WorkflowInfoDTO> fetchWorkflow(Long workflowId) throws PowerJobException { public ResultDTO<WorkflowInfoDTO> fetchWorkflow(Long workflowId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("workflowId", workflowId.toString()) .add("workflowId", workflowId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -307,12 +298,11 @@ public class OhMyClient {
} }
/** /**
* 禁用某个工作流 * Disable Workflow by workflowId
* @param workflowId 工作流ID * @param workflowId workflowId
* @return 标准返回对象 * @return Standard return object
* @throws PowerJobException 异常
*/ */
public ResultDTO<Void> disableWorkflow(Long workflowId) throws PowerJobException { public ResultDTO<Void> disableWorkflow(Long workflowId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("workflowId", workflowId.toString()) .add("workflowId", workflowId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -322,12 +312,11 @@ public class OhMyClient {
} }
/** /**
* 启用某个工作流 * Enable Workflow by workflowId
* @param workflowId workflowId * @param workflowId workflowId
* @return 标准返回对象 * @return Standard return object
* @throws PowerJobException 异常
*/ */
public ResultDTO<Void> enableWorkflow(Long workflowId) throws PowerJobException { public ResultDTO<Void> enableWorkflow(Long workflowId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("workflowId", workflowId.toString()) .add("workflowId", workflowId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -337,12 +326,11 @@ public class OhMyClient {
} }
/** /**
* 删除某个工作流 * Delete Workflow by workflowId
* @param workflowId workflowId * @param workflowId workflowId
* @return 标准返回对象 * @return Standard return object
* @throws PowerJobException 异常
*/ */
public ResultDTO<Void> deleteWorkflow(Long workflowId) throws PowerJobException { public ResultDTO<Void> deleteWorkflow(Long workflowId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("workflowId", workflowId.toString()) .add("workflowId", workflowId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -352,14 +340,13 @@ public class OhMyClient {
} }
/** /**
* 运行工作流 * Run a workflow once
* @param workflowId 工作流ID * @param workflowId workflowId
* @param initParams 启动参数 * @param initParams workflow startup parameters
* @param delayMS 延迟时间，单位毫秒 ms * @param delayMS Delay time（Milliseconds）
* @return 工作流实例ID * @return workflow instanceId
* @throws PowerJobException 异常信息
*/ */
public ResultDTO<Long> runWorkflow(Long workflowId, String initParams, long delayMS) throws PowerJobException { public ResultDTO<Long> runWorkflow(Long workflowId, String initParams, long delayMS) {
FormBody.Builder builder = new FormBody.Builder() FormBody.Builder builder = new FormBody.Builder()
.add("workflowId", workflowId.toString()) .add("workflowId", workflowId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -370,18 +357,17 @@ public class OhMyClient {
String post = postHA(OpenAPIConstant.RUN_WORKFLOW, builder.build()); String post = postHA(OpenAPIConstant.RUN_WORKFLOW, builder.build());
return JSONObject.parseObject(post, LONG_RESULT_TYPE); return JSONObject.parseObject(post, LONG_RESULT_TYPE);
} }
public ResultDTO<Long> runWorkflow(Long workflowId) throws PowerJobException { public ResultDTO<Long> runWorkflow(Long workflowId) {
return runWorkflow(workflowId, null, 0); return runWorkflow(workflowId, null, 0);
} }
/* ************* Workflow Instance API list ************* */ /* ************* Workflow Instance API list ************* */
/** /**
* 停止应用实例 * Stop one workflow instance
* @param wfInstanceId 工作流实例ID * @param wfInstanceId workflow instanceId
* @return true 停止成功 ； false 停止失败 * @return Standard return object
* @throws PowerJobException 异常
*/ */
public ResultDTO<Void> stopWorkflowInstance(Long wfInstanceId) throws PowerJobException { public ResultDTO<Void> stopWorkflowInstance(Long wfInstanceId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("wfInstanceId", wfInstanceId.toString()) .add("wfInstanceId", wfInstanceId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -391,12 +377,11 @@ public class OhMyClient {
} }
/** /**
* 查询任务实例的信息 * Query detail about a workflow instance
* @param wfInstanceId 任务实例ID * @param wfInstanceId workflow instanceId
* @return 任务实例信息 * @return detail about a workflow
* @throws PowerJobException 潜在的异常
*/ */
public ResultDTO<WorkflowInstanceInfoDTO> fetchWorkflowInstanceInfo(Long wfInstanceId) throws PowerJobException { public ResultDTO<WorkflowInstanceInfoDTO> fetchWorkflowInstanceInfo(Long wfInstanceId) {
RequestBody body = new FormBody.Builder() RequestBody body = new FormBody.Builder()
.add("wfInstanceId", wfInstanceId.toString()) .add("wfInstanceId", wfInstanceId.toString())
.add("appId", appId.toString()) .add("appId", appId.toString())
@ -439,6 +424,6 @@ public class OhMyClient {
} }
log.error("[OhMyClient] do post for path: {} failed because of no server available in {}.", path, allAddress); log.error("[OhMyClient] do post for path: {} failed because of no server available in {}.", path, allAddress);
throw new PowerJobException("no server available when send post"); throw new PowerJobException("no server available when send post request");
} }
} }

2
powerjob-common/pom.xml
View File

@ -10,7 +10,7 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-common</artifactId> <artifactId>powerjob-common</artifactId>
<version>3.4.2</version> <version>3.4.3</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<properties> <properties>

2
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/InstanceStatus.java
View File

@ -7,7 +7,7 @@ import lombok.Getter;
import java.util.List; import java.util.List;
/** /**
* 任务运行状态 * Status of the job instance
* *
* @author tjq * @author tjq
* @since 2020/3/17 * @since 2020/3/17

6
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/ProcessorType.java
View File

@ -4,7 +4,7 @@ import lombok.AllArgsConstructor;
import lombok.Getter; import lombok.Getter;
/** /**
* 处理器类型 * Task Processor Type
* *
* @author tjq * @author tjq
* @since 2020/3/23 * @since 2020/3/23
@ -18,8 +18,8 @@ public enum ProcessorType {
PYTHON(3, "Python脚本"), PYTHON(3, "Python脚本"),
JAVA_CONTAINER(4, "Java容器"); JAVA_CONTAINER(4, "Java容器");
private int v; private final int v;
private String des; private final String des;
public static ProcessorType of(int v) { public static ProcessorType of(int v) {
for (ProcessorType type : values()) { for (ProcessorType type : values()) {

8
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/TimeExpressionType.java
View File

@ -7,7 +7,7 @@ import lombok.Getter;
import java.util.List; import java.util.List;
/** /**
* 时间表达式类型 * Scheduling time strategies
* *
* @author tjq * @author tjq
* @since 2020/3/30 * @since 2020/3/30
@ -18,13 +18,13 @@ public enum TimeExpressionType {
API(1), API(1),
CRON(2), CRON(2),
FIX_RATE(3), FIXED_RATE(3),
FIX_DELAY(4), FIXED_DELAY(4),
WORKFLOW(5); WORKFLOW(5);
int v; int v;
public static final List<Integer> frequentTypes = Lists.newArrayList(FIX_RATE.v, FIX_DELAY.v); public static final List<Integer> frequentTypes = Lists.newArrayList(FIXED_RATE.v, FIXED_DELAY.v);
public static TimeExpressionType of(int v) { public static TimeExpressionType of(int v) {
for (TimeExpressionType type : values()) { for (TimeExpressionType type : values()) {

4
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/model/DeployedContainerInfo.java
View File

@ -6,7 +6,7 @@ import lombok.Data;
import lombok.NoArgsConstructor; import lombok.NoArgsConstructor;
/** /**
* The class for deployed container. * Deployed Container Information
* *
* @author tjq * @author tjq
* @since 2020/5/18 * @since 2020/5/18
@ -29,7 +29,7 @@ public class DeployedContainerInfo implements OmsSerializable {
*/ */
private long deployedTime; private long deployedTime;
/** /**
* Address of the server. Report is not required. * No need to report to the server
*/ */
private String workerAddress; private String workerAddress;
} }

5
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/model/InstanceDetail.java
View File

@ -7,7 +7,7 @@ import lombok.NoArgsConstructor;
import java.util.List; import java.util.List;
/** /**
* Detailed info of task instances. * Detailed info of job instances.
* *
* @author tjq * @author tjq
* @since 2020/4/11 * @since 2020/4/11
@ -32,17 +32,14 @@ public class InstanceDetail implements OmsSerializable {
* Status of the task instance. * Status of the task instance.
*/ */
private Integer status; private Integer status;
// 任务执行结果（可能不存在）
/** /**
* Execution result, which may be null. * Execution result, which may be null.
*/ */
private String result; private String result;
// TaskTracker地址
/** /**
* Task tracker address. * Task tracker address.
*/ */
private String taskTrackerAddress; private String taskTrackerAddress;
// 启动参数
/** /**
* Param string that is passed to an instance when it is initialized. * Param string that is passed to an instance when it is initialized.
*/ */

29
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/model/SystemMetrics.java
View File

@ -71,20 +71,17 @@ public class SystemMetrics implements OmsSerializable, Comparable<SystemMetrics>
* @return score * @return score
*/ */
public int calculateScore() { public int calculateScore() {
if (score > 0) {
if (score > 0) { return score;
return score; }
} // Memory is vital to TaskTracker, so we set the multiplier factor as 2.
double memScore = (jvmMaxMemory - jvmUsedMemory) * 2;
// Memory is vital to TaskTracker, so we set the multiplier factor as 2. // Calculate the remaining load of CPU. Multiplier is set as 1.
double memScore = (jvmMaxMemory - jvmUsedMemory) * 2; double cpuScore = cpuProcessors - cpuLoad;
// Calculate the remaining load of CPU. Multiplier is set as 1. // Windows can not fetch CPU load, set cpuScore as 1.
double cpuScore = cpuProcessors - cpuLoad; if (cpuScore > cpuProcessors) {
// Windows can not fetch CPU load, set cpuScore as 1. cpuScore = 1;
if (cpuScore > cpuProcessors) {
cpuScore = 1;
} }
score = (int) (memScore + cpuScore); score = (int) (memScore + cpuScore);
return score; return score;
} }
@ -93,8 +90,8 @@ public class SystemMetrics implements OmsSerializable, Comparable<SystemMetrics>
* Judge if the machine is available. * Judge if the machine is available.
* *
* @param minCPUCores Minimum available CPU cores. * @param minCPUCores Minimum available CPU cores.
* @param minMemorySpace 判断标准之最低可用内存 * @param minMemorySpace Minimum available memory size
* @param minDiskSpace Minimum disk space 判断标准之最低可用磁盘空间 * @param minDiskSpace Minimum disk space
* @return {@code boolean} whether the machine is available. * @return {@code boolean} whether the machine is available.
*/ */
public boolean available(double minCPUCores, double minMemorySpace, double minDiskSpace) { public boolean available(double minCPUCores, double minMemorySpace, double minDiskSpace) {
@ -106,8 +103,8 @@ public class SystemMetrics implements OmsSerializable, Comparable<SystemMetrics>
return false; return false;
} }
// Negative number means being unable to fetch CPU info, return true.
// 0 indicates the CPU is free, which is the optimal condition. // 0 indicates the CPU is free, which is the optimal condition.
// Negative number means being unable to fetch CPU info, return true.
if (cpuLoad <= 0 || minCPUCores <= 0) { if (cpuLoad <= 0 || minCPUCores <= 0) {
return true; return true;
} }

8
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/request/http/SaveJobInfoRequest.java
View File

@ -18,7 +18,7 @@ import java.util.List;
public class SaveJobInfoRequest { public class SaveJobInfoRequest {
/** /**
* id of the job. set null to save or non-null to update the job. * id of the job. set null to create or non-null to update the job.
*/ */
private Long id; private Long id;
/* ************************** Base info of related job. ************************** */ /* ************************** Base info of related job. ************************** */
@ -105,11 +105,6 @@ public class SaveJobInfoRequest {
*/ */
private double minDiskSpace = 0; private double minDiskSpace = 0;
/**
* {@code 1} indicates that the worker node is running well,
* {@code 2} indicates that the worker node has been inactive
* and future tasks will not be assigned to the node.
*/
private boolean enable = true; private boolean enable = true;
@ -117,6 +112,7 @@ public class SaveJobInfoRequest {
/** /**
* Designated PowerJob-worker nodes. Blank value indicates that there is * Designated PowerJob-worker nodes. Blank value indicates that there is
* no limit. Non-blank value means to run the corresponding machine(s) only. * no limit. Non-blank value means to run the corresponding machine(s) only.
* example: 192.168.1.1:27777,192.168.1.2:27777
*/ */
private String designatedWorkers; private String designatedWorkers;
/** /**

2
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/response/InstanceInfoDTO.java
View File

@ -6,7 +6,7 @@ import lombok.Data;
import java.util.Date; import java.util.Date;
/** /**
* instanceInfo 对外输出对象 * instanceInfo Network transmission object
* *
* @author tjq * @author tjq
* @since 2020/5/14 * @since 2020/5/14

4
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/response/ResultDTO.java
View File

@ -7,7 +7,7 @@ import lombok.ToString;
import org.apache.commons.lang3.exception.ExceptionUtils; import org.apache.commons.lang3.exception.ExceptionUtils;
/** /**
* 请求返回的结果对象 * The result object returned by the request
* *
* @author tjq * @author tjq
* @since 2020/3/30 * @since 2020/3/30
@ -18,9 +18,7 @@ import org.apache.commons.lang3.exception.ExceptionUtils;
public class ResultDTO<T> implements OmsSerializable { public class ResultDTO<T> implements OmsSerializable {
private boolean success; private boolean success;
// 数据（success为 true 时存在）
private T data; private T data;
// 错误信息（success为 false 时存在）
private String message; private String message;
public static <T> ResultDTO<T> success(T data) { public static <T> ResultDTO<T> success(T data) {

4
powerjob-server/pom.xml
View File

@ -10,13 +10,13 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-server</artifactId> <artifactId>powerjob-server</artifactId>
<version>3.4.2</version> <version>3.4.3</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<properties> <properties>
<swagger.version>2.9.2</swagger.version> <swagger.version>2.9.2</swagger.version>
<springboot.version>2.3.4.RELEASE</springboot.version> <springboot.version>2.3.4.RELEASE</springboot.version>
<powerjob.common.version>3.4.2</powerjob.common.version> <powerjob.common.version>3.4.3</powerjob.common.version>
<!-- MySQL version that corresponds to spring-boot-dependencies version. --> <!-- MySQL version that corresponds to spring-boot-dependencies version. -->
<mysql.version>8.0.19</mysql.version> <mysql.version>8.0.19</mysql.version>
<ojdbc.version>19.7.0.0</ojdbc.version> <ojdbc.version>19.7.0.0</ojdbc.version>

5
powerjob-server/src/main/java/com/github/kfcfans/powerjob/server/OhMyApplication.java
View File

@ -7,7 +7,7 @@ import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableScheduling; import org.springframework.scheduling.annotation.EnableScheduling;
/** /**
* SpringBoot entry. * powerjob-server entry
* *
* @author tjq * @author tjq
* @since 2020/3/29 * @since 2020/3/29
@ -26,10 +26,9 @@ public class OhMyApplication {
public static void main(String[] args) { public static void main(String[] args) {
// Print tips before starting.
pre(); pre();
// Init ActorSystem // Init ActorSystem first
OhMyServer.init(); OhMyServer.init();
// Start SpringBoot application. // Start SpringBoot application.

2
powerjob-server/src/main/java/com/github/kfcfans/powerjob/server/common/config/SwaggerConfig.java
View File

@ -44,7 +44,7 @@ public class SwaggerConfig {
.title("PowerJob") .title("PowerJob")
.description("Distributed scheduling and computing framework.") .description("Distributed scheduling and computing framework.")
.license("Apache Licence 2") .license("Apache Licence 2")
.termsOfServiceUrl("https://github.com/KFCFans/PowerJob") .termsOfServiceUrl("https://github.com/PowerJob/PowerJob")
.version(version) .version(version)
.build(); .build();

4
powerjob-server/src/main/java/com/github/kfcfans/powerjob/server/service/ValidateService.java
View File

@ -33,8 +33,8 @@ public class ValidateService {
case API: return Lists.newArrayList(OmsConstant.NONE); case API: return Lists.newArrayList(OmsConstant.NONE);
case WORKFLOW: return Lists.newArrayList("VALID: depends on workflow"); case WORKFLOW: return Lists.newArrayList("VALID: depends on workflow");
case CRON: return calculateCronExpression(timeExpression); case CRON: return calculateCronExpression(timeExpression);
case FIX_RATE: return calculateFixRate(timeExpression); case FIXED_RATE: return calculateFixRate(timeExpression);
case FIX_DELAY: return Lists.newArrayList("VALID: depends on execution cost time"); case FIXED_DELAY: return Lists.newArrayList("VALID: depends on execution cost time");
} }
// impossible // impossible
return Collections.emptyList(); return Collections.emptyList();

8
powerjob-server/src/main/resources/application-daily.properties
View File

@ -14,7 +14,8 @@ spring.datasource.core.hikari.minimum-idle=5
oms.mongodb.enable=true oms.mongodb.enable=true
spring.data.mongodb.uri=mongodb+srv://zqq:No1Bug2Please3!@cluster0.wie54.gcp.mongodb.net/powerjob_daily?retryWrites=true&w=majority spring.data.mongodb.uri=mongodb+srv://zqq:No1Bug2Please3!@cluster0.wie54.gcp.mongodb.net/powerjob_daily?retryWrites=true&w=majority
####### Email properties(Comment out the mail properties if you do not have needs) ####### ####### Email properties(Non-core configuration properties) #######
####### Delete the following code to disable the mail #######
spring.mail.host=smtp.163.com spring.mail.host=smtp.163.com
spring.mail.username=zqq@163.com spring.mail.username=zqq@163.com
spring.mail.password=GOFZPNARMVKCGONV spring.mail.password=GOFZPNARMVKCGONV
@ -22,7 +23,8 @@ spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true spring.mail.properties.mail.smtp.starttls.required=true
####### DingTalk properties(Comment out the DingTalk properties if you do not have needs) ####### ####### DingTalk properties(Non-core configuration properties) #######
####### Delete the following code to disable the DingTalk #######
oms.alarm.ding.app-key=dingauqwkvxxnqskknfv oms.alarm.ding.app-key=dingauqwkvxxnqskknfv
oms.alarm.ding.app-secret=XWrEPdAZMPgJeFtHuL0LH73LRj-74umF2_0BFcoXMfvnX0pCQvt0rpb1JOJU_HLl oms.alarm.ding.app-secret=XWrEPdAZMPgJeFtHuL0LH73LRj-74umF2_0BFcoXMfvnX0pCQvt0rpb1JOJU_HLl
oms.alarm.ding.agent-id=847044348 oms.alarm.ding.agent-id=847044348
@ -35,6 +37,6 @@ oms.container.retention.remote=-1
####### Cache properties ####### ####### Cache properties #######
oms.instance.metadata.cache.size=1024 oms.instance.metadata.cache.size=1024
####### Threshold in fetching server(0~100). 100 means full detection of server, in which ####### ####### Threshold in precise fetching server(0~100). 100 means full detection of server, in which #######
####### split-brain could be avoided while performance overhead would increase. ####### ####### split-brain could be avoided while performance overhead would increase. #######
oms.accurate.select.server.percentage = 50 oms.accurate.select.server.percentage = 50

8
powerjob-server/src/main/resources/application-pre.properties
View File

@ -14,7 +14,8 @@ spring.datasource.core.hikari.minimum-idle=5
oms.mongodb.enable=true oms.mongodb.enable=true
spring.data.mongodb.uri=mongodb://remotehost:27017/powerjob-pre spring.data.mongodb.uri=mongodb://remotehost:27017/powerjob-pre
####### Email properties(Comment out the mail properties if you do not have needs) ####### ####### Email properties(Non-core configuration properties) #######
####### Delete the following code to disable the mail #######
spring.mail.host=smtp.qq.com spring.mail.host=smtp.qq.com
spring.mail.username=zqq spring.mail.username=zqq
spring.mail.password=qqz spring.mail.password=qqz
@ -22,7 +23,8 @@ spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true spring.mail.properties.mail.smtp.starttls.required=true
####### DingTalk properties(Comment out the DingTalk properties if you do not have needs) ####### ####### DingTalk properties(Non-core configuration properties) #######
####### Delete the following code to disable the DingTalk #######
oms.alarm.ding.app-key=dingauqwkvxxnqskknfv oms.alarm.ding.app-key=dingauqwkvxxnqskknfv
oms.alarm.ding.app-secret=XWrEPdAZMPgJeFtHuL0LH73LRj-74umF2_0BFcoXMfvnX0pCQvt0rpb1JOJU_HLl oms.alarm.ding.app-secret=XWrEPdAZMPgJeFtHuL0LH73LRj-74umF2_0BFcoXMfvnX0pCQvt0rpb1JOJU_HLl
oms.alarm.ding.agent-id=847044348 oms.alarm.ding.agent-id=847044348
@ -35,6 +37,6 @@ oms.container.retention.remote=-1
####### Cache properties ####### ####### Cache properties #######
oms.instance.metadata.cache.size=1024 oms.instance.metadata.cache.size=1024
####### Threshold in fetching server(0~100). 100 means full detection of server, in which ####### ####### Threshold in precise fetching server(0~100). 100 means full detection of server, in which #######
####### split-brain could be avoided while performance overhead would increase. ####### ####### split-brain could be avoided while performance overhead would increase. #######
oms.accurate.select.server.percentage = 50 oms.accurate.select.server.percentage = 50

8
powerjob-server/src/main/resources/application-product.properties
View File

@ -14,7 +14,8 @@ spring.datasource.core.hikari.minimum-idle=5
oms.mongodb.enable=true oms.mongodb.enable=true
spring.data.mongodb.uri=mongodb://localhost:27017/powerjob-product spring.data.mongodb.uri=mongodb://localhost:27017/powerjob-product
####### Email properties(Comment out the mail properties if you do not have needs) ####### ####### Email properties(Non-core configuration properties) #######
####### Delete the following code to disable the mail #######
spring.mail.host=smtp.qq.com spring.mail.host=smtp.qq.com
spring.mail.username=zqq spring.mail.username=zqq
spring.mail.password=qqz spring.mail.password=qqz
@ -22,7 +23,8 @@ spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true spring.mail.properties.mail.smtp.starttls.required=true
####### DingTalk properties(Comment out the DingTalk properties if you do not have needs) ####### ####### DingTalk properties(Non-core configuration properties) #######
####### Delete the following code to disable the DingTalk #######
oms.alarm.ding.app-key= oms.alarm.ding.app-key=
oms.alarm.ding.app-secret= oms.alarm.ding.app-secret=
oms.alarm.ding.agent-id= oms.alarm.ding.agent-id=
@ -35,6 +37,6 @@ oms.container.retention.remote=-1
####### Cache properties ####### ####### Cache properties #######
oms.instance.metadata.cache.size=2048 oms.instance.metadata.cache.size=2048
####### Threshold in fetching server(0~100). 100 means full detection of server, in which ####### ####### Threshold in precise fetching server(0~100). 100 means full detection of server, in which #######
####### split-brain could be avoided while performance overhead would increase. ####### ####### split-brain could be avoided while performance overhead would increase. #######
oms.accurate.select.server.percentage = 50 oms.accurate.select.server.percentage = 50

2
powerjob-server/src/main/resources/banner.txt
View File

@ -10,5 +10,5 @@ ${AnsiColor.GREEN}
${AnsiColor.BRIGHT_RED} ${AnsiColor.BRIGHT_RED}
* Maintainer: tengjiqi@gmail.com & PowerJob-Team * Maintainer: tengjiqi@gmail.com & PowerJob-Team
* OfficialWebsite: http://www.powerjob.tech/ * OfficialWebsite: http://www.powerjob.tech/
* SourceCode: https://github.com/KFCFans/PowerJob * SourceCode: https://github.com/PowerJob/PowerJob
* PoweredBy: SpringBoot${spring-boot.formatted-version} & Akka (v2.6.4) * PoweredBy: SpringBoot${spring-boot.formatted-version} & Akka (v2.6.4)

2
powerjob-server/src/main/resources/static/js/5.js
View File

File diff suppressed because one or more lines are too long

2
powerjob-server/src/main/resources/static/js/6.js
View File

File diff suppressed because one or more lines are too long

2
powerjob-server/src/main/resources/static/js/9.js
View File

File diff suppressed because one or more lines are too long

2
powerjob-server/src/main/resources/static/js/app.js
View File

File diff suppressed because one or more lines are too long

5
powerjob-worker-agent/pom.xml
View File

@ -10,12 +10,12 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-worker-agent</artifactId> <artifactId>powerjob-worker-agent</artifactId>
<version>3.4.2</version> <version>3.4.3</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<properties> <properties>
<powerjob.worker.version>3.4.2</powerjob.worker.version> <powerjob.worker.version>3.4.3</powerjob.worker.version>
<logback.version>1.2.3</logback.version> <logback.version>1.2.3</logback.version>
<picocli.version>4.3.2</picocli.version> <picocli.version>4.3.2</picocli.version>
@ -50,6 +50,7 @@
</dependencies> </dependencies>
<!-- 谁说SpringBoot的打包插件只能给SpringBoot用的？省的我写一堆配置还有BUG... -->
<build> <build>
<plugins> <plugins>
<plugin> <plugin>

4
powerjob-worker-agent/src/main/java/com/github/kfcfans/powerjob/worker/MainApplication.java
View File

@ -11,13 +11,13 @@ import picocli.CommandLine.Command;
import picocli.CommandLine.Option; import picocli.CommandLine.Option;
/** /**
* 启动类 * powerjob-worker-agent entry
* *
* @author tjq * @author tjq
* @since 2020/5/20 * @since 2020/5/20
*/ */
@Slf4j @Slf4j
@Command(name = "OhMyAgent", mixinStandardHelpOptions = true, version = "1.2.0", description = "OhMyScheduler-Worker agent") @Command(name = "OhMyAgent", mixinStandardHelpOptions = true, version = "3.4.3", description = "powerjob-worker agent")
public class MainApplication implements Runnable { public class MainApplication implements Runnable {
@Option(names = {"-a", "--app"}, description = "worker-agent's name", required = true) @Option(names = {"-a", "--app"}, description = "worker-agent's name", required = true)

4
powerjob-worker-samples/pom.xml
View File

@ -10,11 +10,11 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-worker-samples</artifactId> <artifactId>powerjob-worker-samples</artifactId>
<version>3.4.2</version> <version>3.4.3</version>
<properties> <properties>
<springboot.version>2.2.6.RELEASE</springboot.version> <springboot.version>2.2.6.RELEASE</springboot.version>
<powerjob.worker.starter.version>3.4.2</powerjob.worker.starter.version> <powerjob.worker.starter.version>3.4.3</powerjob.worker.starter.version>
<fastjson.version>1.2.68</fastjson.version> <fastjson.version>1.2.68</fastjson.version>
<!-- 部署时跳过该module --> <!-- 部署时跳过该module -->

2
powerjob-worker-samples/src/main/java/com/github/kfcfans/powerjob/samples/tester/StopInstanceTester.java
View File

@ -7,7 +7,7 @@ import org.springframework.stereotype.Component;
/** /**
* 测试用户反馈的无法停止实例的问题 * 测试用户反馈的无法停止实例的问题
* https://github.com/KFCFans/PowerJob/issues/37 * https://github.com/PowerJob/PowerJob/issues/37
* *
* @author tjq * @author tjq
* @since 2020/7/30 * @since 2020/7/30

4
powerjob-worker-spring-boot-starter/pom.xml
View File

@ -10,11 +10,11 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-worker-spring-boot-starter</artifactId> <artifactId>powerjob-worker-spring-boot-starter</artifactId>
<version>3.4.2</version> <version>3.4.3</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<properties> <properties>
<powerjob.worker.version>3.4.2</powerjob.worker.version> <powerjob.worker.version>3.4.3</powerjob.worker.version>
<springboot.version>2.2.6.RELEASE</springboot.version> <springboot.version>2.2.6.RELEASE</springboot.version>
</properties> </properties>

4
powerjob-worker-spring-boot-starter/src/main/java/com/github/kfcfans/powerjob/worker/autoconfigure/PowerJobProperties.java
View File

@ -130,9 +130,7 @@ public class PowerJobProperties {
private int maxResultLength = 8096; private int maxResultLength = 8096;
/** /**
* If test mode is set as true, Powerjob-worker no longer connects to the server or validates appName. * If test mode is set as true, Powerjob-worker no longer connects to the server or validates appName.
* Test mode is used for conditions that your worker does not need to run the codes, i.e. when you * Test mode is used for conditions that your have no powerjob-server in your develop env so you can't startup the application
* write junit tests in local environment. {@code true} means test mode is enabled. {@code false} means
* normal mode is applied.
*/ */
private boolean enableTestMode = false; private boolean enableTestMode = false;
} }

4
powerjob-worker/pom.xml
View File

@ -10,12 +10,12 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-worker</artifactId> <artifactId>powerjob-worker</artifactId>
<version>3.4.2</version> <version>3.4.3</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<properties> <properties>
<spring.version>5.2.4.RELEASE</spring.version> <spring.version>5.2.4.RELEASE</spring.version>
<powerjob.common.version>3.4.2</powerjob.common.version> <powerjob.common.version>3.4.3</powerjob.common.version>
<h2.db.version>1.4.200</h2.db.version> <h2.db.version>1.4.200</h2.db.version>
<hikaricp.version>3.4.2</hikaricp.version> <hikaricp.version>3.4.2</hikaricp.version>
<junit.version>5.6.1</junit.version> <junit.version>5.6.1</junit.version>

32
powerjob-worker/src/main/java/com/github/kfcfans/powerjob/worker/common/OhMyConfig.java
View File

@ -10,7 +10,7 @@ import lombok.Setter;
import java.util.List; import java.util.List;
/** /**
* Worker 配置文件 * The powerjob-worker's configuration
* *
* @author tjq * @author tjq
* @since 2020/3/16 * @since 2020/3/16
@ -19,34 +19,38 @@ import java.util.List;
@Setter @Setter
public class OhMyConfig { public class OhMyConfig {
/** /**
* 应用名称 * AppName, recommend to use the name of this project
* Applications should be registered by powerjob-console in advance to prevent error.
*/ */
private String appName; private String appName;
/** /**
* 启动端口 * Worker port
* Random port is enabled when port is set with non-positive number.
*/ */
private int port = RemoteConstant.DEFAULT_WORKER_PORT; private int port = RemoteConstant.DEFAULT_WORKER_PORT;
/** /**
* 调度服务器地址，ip:port 或 域名 * Address of powerjob-server node(s)
* Do not mistake for ActorSystem port. Do not add any prefix, i.e. http://.
*/ */
private List<String> serverAddress = Lists.newArrayList(); private List<String> serverAddress = Lists.newArrayList();
/** /**
* 本地持久化方式，默认使用磁盘 * Max length of response result. Result that is longer than the value will be truncated.
*/ * {@link ProcessResult} max length for #msg
private StoreStrategy storeStrategy = StoreStrategy.DISK;
/**
* 最大返回值长度，超过会被截断
* {@link ProcessResult}#msg 的最大长度
*/ */
private int maxResultLength = 8096; private int maxResultLength = 8096;
/** /**
* 用户自定义上下文对象，该值会被透传到 TaskContext#userContext 属性 * User-defined context object, which is passed through to the TaskContext#userContext property
* 使用场景：容器脚本Java处理器需要使用oms-worker宿主应用的Spring Bean，可在此处传入 ApplicationContext，在Processor中获取 bean * Usage Scenarios: The container Java processor needs to use the Spring bean of the host application, where you can pass in the ApplicationContext and get the bean in the Processor
*/ */
private Object userContext; private Object userContext;
/** /**
* 启动测试模式，true情况下，不再尝试连接 server 并验证appName * Internal persistence method, DISK or MEMORY
* true -> 用于本地写单元测试调试； false -> 默认值，标准模式 * Normally you don't need to care about this configuration
*/
private StoreStrategy storeStrategy = StoreStrategy.DISK;
/**
* If test mode is set as true, Powerjob-worker no longer connects to the server or validates appName.
* Test mode is used for conditions that your have no powerjob-server in your develop env so you can't startup the application
*/ */
private boolean enableTestMode = false; private boolean enableTestMode = false;
} }

2
powerjob-worker/src/main/java/com/github/kfcfans/powerjob/worker/common/PowerBannerPrinter.java
View File

@ -24,7 +24,7 @@ public final class PowerBannerPrinter {
"\n" + "\n" +
"* Maintainer: tengjiqi@gmail.com & PowerJob-Team\n" + "* Maintainer: tengjiqi@gmail.com & PowerJob-Team\n" +
"* OfficialWebsite: http://www.powerjob.tech/\n" + "* OfficialWebsite: http://www.powerjob.tech/\n" +
"* SourceCode: https://github.com/KFCFans/PowerJob\n" + "* SourceCode: https://github.com/PowerJob/PowerJob\n" +
"\n"; "\n";
public static void print() { public static void print() {

4
powerjob-worker/src/main/java/com/github/kfcfans/powerjob/worker/common/constants/StoreStrategy.java
View File

@ -1,6 +1,7 @@
package com.github.kfcfans.powerjob.worker.common.constants; package com.github.kfcfans.powerjob.worker.common.constants;
import lombok.AllArgsConstructor; import lombok.AllArgsConstructor;
import lombok.Getter;
/** /**
* 持久化策略 * 持久化策略
@ -8,11 +9,12 @@ import lombok.AllArgsConstructor;
* @author tjq * @author tjq
* @since 2020/4/14 * @since 2020/4/14
*/ */
@Getter
@AllArgsConstructor @AllArgsConstructor
public enum StoreStrategy { public enum StoreStrategy {
DISK("磁盘"), DISK("磁盘"),
MEMORY("内存"); MEMORY("内存");
private String des; private final String des;
} }

6
powerjob-worker/src/main/java/com/github/kfcfans/powerjob/worker/core/tracker/task/FrequentTaskTracker.java
View File

@ -88,7 +88,7 @@ public class FrequentTaskTracker extends TaskTracker {
// 2. 启动任务发射器 // 2. 启动任务发射器
launcher = new Launcher(); launcher = new Launcher();
if (timeExpressionType == TimeExpressionType.FIX_RATE) { if (timeExpressionType == TimeExpressionType.FIXED_RATE) {
// 固定频率需要设置最小间隔 // 固定频率需要设置最小间隔
if (timeParams < MIN_INTERVAL) { if (timeParams < MIN_INTERVAL) {
throw new PowerJobException("time interval too small, please set the timeExpressionInfo >= 1000"); throw new PowerJobException("time interval too small, please set the timeExpressionInfo >= 1000");
@ -172,7 +172,7 @@ public class FrequentTaskTracker extends TaskTracker {
// 判断是否超出最大执行实例数 // 判断是否超出最大执行实例数
if (maxInstanceNum > 0) { if (maxInstanceNum > 0) {
if (timeExpressionType == TimeExpressionType.FIX_RATE) { if (timeExpressionType == TimeExpressionType.FIXED_RATE) {
if (subInstanceId2TimeHolder.size() > maxInstanceNum) { if (subInstanceId2TimeHolder.size() > maxInstanceNum) {
log.warn("[FQTaskTracker-{}] cancel to launch the subInstance({}) due to too much subInstance is running.", instanceId, subInstanceId); log.warn("[FQTaskTracker-{}] cancel to launch the subInstance({}) due to too much subInstance is running.", instanceId, subInstanceId);
processFinishedSubInstance(subInstanceId, false, "TOO_MUCH_INSTANCE"); processFinishedSubInstance(subInstanceId, false, "TOO_MUCH_INSTANCE");
@ -368,7 +368,7 @@ public class FrequentTaskTracker extends TaskTracker {
taskPersistenceService.deleteAllSubInstanceTasks(instanceId, subInstanceId); taskPersistenceService.deleteAllSubInstanceTasks(instanceId, subInstanceId);
// FIX_DELAY 则调度下次任务 // FIX_DELAY 则调度下次任务
if (timeExpressionType == TimeExpressionType.FIX_DELAY) { if (timeExpressionType == TimeExpressionType.FIXED_DELAY) {
scheduledPool.schedule(launcher, timeParams, TimeUnit.MILLISECONDS); scheduledPool.schedule(launcher, timeParams, TimeUnit.MILLISECONDS);
} }
} }

4
powerjob-worker/src/main/java/com/github/kfcfans/powerjob/worker/core/tracker/task/TaskTracker.java
View File

@ -115,8 +115,8 @@ public abstract class TaskTracker {
try { try {
TimeExpressionType timeExpressionType = TimeExpressionType.valueOf(req.getTimeExpressionType()); TimeExpressionType timeExpressionType = TimeExpressionType.valueOf(req.getTimeExpressionType());
switch (timeExpressionType) { switch (timeExpressionType) {
case FIX_RATE: case FIXED_RATE:
case FIX_DELAY:return new FrequentTaskTracker(req); case FIXED_DELAY:return new FrequentTaskTracker(req);
default:return new CommonTaskTracker(req); default:return new CommonTaskTracker(req);
} }
} catch (Exception e) { } catch (Exception e) {

4
powerjob-worker/src/test/java/com/github/kfcfans/powerjob/FrequentTaskTrackerTest.java
View File

@ -41,13 +41,13 @@ public class FrequentTaskTrackerTest {
@Test @Test
public void testFixRateJob() throws Exception { public void testFixRateJob() throws Exception {
remoteTaskTracker.tell(TestUtils.genServerScheduleJobReq(ExecuteType.STANDALONE, TimeExpressionType.FIX_RATE), null); remoteTaskTracker.tell(TestUtils.genServerScheduleJobReq(ExecuteType.STANDALONE, TimeExpressionType.FIXED_RATE), null);
Thread.sleep(5000000); Thread.sleep(5000000);
} }
@Test @Test
public void testFixDelayJob() throws Exception { public void testFixDelayJob() throws Exception {
remoteTaskTracker.tell(TestUtils.genServerScheduleJobReq(ExecuteType.MAP_REDUCE, TimeExpressionType.FIX_DELAY), null); remoteTaskTracker.tell(TestUtils.genServerScheduleJobReq(ExecuteType.MAP_REDUCE, TimeExpressionType.FIXED_DELAY), null);
Thread.sleep(5000000); Thread.sleep(5000000);
} }
} }

4
powerjob-worker/src/test/java/com/github/kfcfans/powerjob/TestUtils.java
View File

@ -32,8 +32,8 @@ public class TestUtils {
req.setTimeExpressionType(timeExpressionType.name()); req.setTimeExpressionType(timeExpressionType.name());
switch (timeExpressionType) { switch (timeExpressionType) {
case CRON:req.setTimeExpression("0 * * * * ? "); case CRON:req.setTimeExpression("0 * * * * ? ");
case FIX_RATE: case FIXED_RATE:
case FIX_DELAY:req.setTimeExpression("5000"); case FIXED_DELAY:req.setTimeExpression("5000");
} }
switch (executeType) { switch (executeType) {