third_codes/PowerJob
1
0
Fork 1
mirror of https://github.com/PowerJob/PowerJob.git synced 2025-07-17 00:00:04 +08:00
Code Issues Packages Projects Releases Wiki Activity

release: v3.4.3

Browse Source
This commit is contained in:
tjq 2021-01-10 18:16:05 +08:00
commit 2ac9cb44e9
60 changed files with 595 additions and 415 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

52
README.md
View File

@ -5,38 +5,37 @@ English | [简体中文](./README_zhCN.md)
</p>
<p align="center">
<a href="https://github.com/KFCFans/PowerJob/actions"><img src="https://github.com/KFCFans/PowerJob/workflows/Java%20CI%20with%20Maven/badge.svg?branch=master" alt="actions"></a>
<a href="https://github.com/PowerJob/PowerJob/actions"><img src="https://github.com/PowerJob/PowerJob/workflows/Java%20CI%20with%20Maven/badge.svg?branch=master" alt="actions"></a>
<a href="https://search.maven.org/search?q=com.github.kfcfans"><img alt="Maven Central" src="https://img.shields.io/maven-central/v/com.github.kfcfans/powerjob-worker"></a>
<a href="https://github.com/KFCFans/PowerJob/releases"><img alt="GitHub release (latest SemVer)" src="https://img.shields.io/github/v/release/kfcfans/powerjob?color=%23E59866"></a>
<a href="https://github.com/KFCFans/PowerJob/blob/master/LICENSE"><img src="https://img.shields.io/github/license/KFCFans/PowerJob" alt="LICENSE"></a>
<a href="https://github.com/PowerJob/PowerJob/releases"><img alt="GitHub release (latest SemVer)" src="https://img.shields.io/github/v/release/kfcfans/powerjob?color=%23E59866"></a>
<a href="https://github.com/PowerJob/PowerJob/blob/master/LICENSE"><img src="https://img.shields.io/github/license/KFCFans/PowerJob" alt="LICENSE"></a>
</p>
- Have you ever wondered how cron jobs could be organized orderly?
- Have you ever felt upset when scheduling tasks suddenly terminated without any warning?
- Have you ever felt helpless when batches of business tasks require handling?
- Have you ever felt depressed about tasks that carry with complex dependencies?
- Have you ever felt upset about tasks that carry with complex dependencies?
- Have you ever felt helpless when scheduling tasks suddenly terminated without any warning?
- Have you ever felt depressed when batches of business tasks need to be processed in a distributed manner?
Well, PowerJob is there for you, it is the choice of a new generation. It is a powerful, business-oriented scheduling framework that provides distributed computing ability. Based on Akka architecture, it makes everything with scheduling easier. Just with several steps, PowerJob could be deployed and work for you!
# Introduction
### Features
- Simple to use: PowerJob provides a friendly front-end Web that allows developers to visually manage tasks (Create, Read, Update and Delete), monitor tasks, and view logs online.
- Simple to use: PowerJob provides a friendly front-end Web that allows developers to visually manage tasks, monitor tasks, and view logs online.
- Complete timing strategy: PowerJob supports four different scheduling strategies, including CRON expression, fixed frequency timing, fixed delay timing as well as the Open API.
- Various execution modes: PowerJob supports four execution modes: stand-alone, broadcast, Map, and MapReduce. It's worth mentioning the Map and MapReduce modes. With several lines of codes, developers could take full advantage of PowerJob's distributed computing ability.
- Complete workflow support. PowerJob supports DAG(Directed acyclic graph) based online task configuration. Developers could arrange tasks on the console, while data could be transferred among tasks on the flow.
- Various execution modes: PowerJob supports four execution modes: stand-alone, broadcast, Map, and MapReduce. **It's worth mentioning the Map and MapReduce modes. With several lines of codes, developers could take full advantage of PowerJob's distributed computing ability**.
- Complete workflow support: PowerJob supports DAG(Directed acyclic graph) based online task configuration. Developers could arrange tasks on the console, while data could be transferred among tasks on the flow.
- Extensive executor support: PowerJob supports multiple processors, including Spring Beans, ordinary Java objects, Shell, Python and so on.
- Simple in dependency: PowerJob aims to be simple in dependency. The only dependency is merely database (MySQL / Oracle / MS SQLServer ...), with MongoDB being the extra dependency for storing large log files online.
- Simple in dependency: PowerJob aims to be simple in dependency. The only dependency is merely database (MySQL / Oracle / MS SQLServer ...), with MongoDB being the extra dependency for storing large log files.
- High availability and performance: Unlike traditional job-scheduling frameworks that rely on database locks, PowerJob server is lock-free. PowerJob supports unlimited horizontal expansion. It's easy to achieve high availability and performance by deploying as many PowerJob server instances as you need.
- Quick failover and recovery support: Whenever any task failed, PowerJob server would retry according to the configured strategy. As long as there were enough nodes in the cluster, the failed tasks could execute successfully finally.
- Convenient to run and maintain: PowerJob supports online logging. Logs generated by the worker would be transferred and displayed on the console instantly, therefore reducing the cost of debugging and improving the efficiency significantly.
### Applicable scenes
- Scenarios with timed tasks: such as full synchronization of data at midnight, generating business reports at desired time.
- Scenarios that require all machines to run tasks simultaneously: such as log cleanup.
- Scenarios that require distributed processing: For example, a large amount of data requires updating, while the stand-alone execution takes quite a lot of time. The Map/MapReduce mode could be applied in which the workers would join the cluster for PowerJob server to dispatch, to speed up the time-consuming process, therefore improving the computing ability of the whole cluster.
- Scenarios with delayed tasks: For instance, disposal of overdue orders.
- **Scenarios with delayed tasks**: For instance, disposal of overdue orders.
### Design goals
@ -51,17 +50,17 @@ Application password: 123
### Comparison with similar products
| | QuartZ | xxl-job | SchedulerX 2.0 | PowerJob |
| ---------------------------------- | --------------------------------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| Timing type | CRON | CRON | CRON, fixed frequency, fixed delay, OpenAPI | **CRON, fixed frequency, fixed delay, OpenAPI** |
| Task type | Built-in Java | Built-in Java, GLUE Java, Shell, Python and other scripts | Built-in Java, external Java (FatJar), Shell, Python and other scripts | **Built-in Java, external Java (container), Shell, Python and other scripts** |
| Distributed strategy | Unsupported | Static sharding | MapReduce dynamic sharding | **MapReduce dynamic sharding** |
| Online task management | Unsupported | Supported | Supported | **Supported** |
| Online logging | Unsupported | Supported | Unsupported | **Supported** |
| Scheduling methods and performance | Based on database lock, there is a performance bottleneck | Based on database lock, there is a performance bottleneck | Unknown | **Lock-free design, powerful performance without upper limit** |
| Alarm monitoring | Unsupported | Email | SMS | **Email, WebHook, DingTalk. An interface is provided for customization.** |
| System dependence | Any relational database (MySQL, Oracle ...) supported by JDBC | MySQL | RMB (Public Beta version for free, hey, helping to promote) | **Any relational database (MySQL, Oracle ...) supported by Spring Data Jpa** |
| workflow | Unsupported | Unsupported | Supported | **Supported** |
| | QuartZ | PowerJob |
| ---------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------ |
| Timing type | CRON | **CRON, fixed frequency, fixed delay, OpenAPI** |
| Task type | Built-in Java | **Built-in Java, external Java (JVM Container), Shell, Python and other scripts** |
| Distributed strategy | Unsupported | **MapReduce dynamic sharding** |
| Online task management | Unsupported | **Supported** |
| Online logging | Unsupported | **Supported** |
| Scheduling methods and performance | Based on database lock, there is a performance bottleneck | **Lock-free design, high performance without upper limit** |
| Alarm monitoring | Unsupported | **Email, WebHook, DingTalk. An interface is provided for customization.** |
| System dependence | Any relational database (MySQL, Oracle ...) supported by JDBC | **Any relational database (MySQL, Oracle ...) supported by Spring Data Jpa** |
| workflow | Unsupported | **Supported** |
# Document
**[Docs](https://www.yuque.com/powerjob/en/introduce)**
@ -69,7 +68,7 @@ Application password: 123
**[中文文档](https://www.yuque.com/powerjob/product)**
# User Registration
[Click to register as PowerJob user and contribute to PowerJob!](https://github.com/KFCFans/PowerJob/issues/6)
[Click to register as PowerJob user and contribute to PowerJob!](https://github.com/PowerJob/PowerJob/issues/6)
ღ( ´・ᴗ・\` )ღ Many thanks to the following registered users. ღ( ´・ᴗ・\` )ღ
<p style="text-align: center">
<img src="https://raw.githubusercontent.com/KFCFans/PowerJob/master/others/images/user.png" alt="PowerJob User" title="PowerJob User"/>
@ -77,9 +76,8 @@ Application password: 123
# Others
- PowerJob is permanently open source software(Apache License, Version 2.0), please feel free to try, deploy and put into production!
- Author of PowerJob (@KFCFans) has abundant time for maintenance, and is willing to provide technical support if you have needs!
- Welcome to the Gitter Community: [LINK](https://gitter.im/PowerJob/community)
- PowerJob is permanently open source software(Apache License, Version 2.0), please feel free to try, deploy and put into production!
- Welcome to contribute to PowerJob, both Pull Requests and Issues are precious.
- Please STAR PowerJob if it is valuable. ~ = ￣ω￣ =
- Do you need any help or want to propose suggestions? Please raise Github issues or contact the Author @KFCFans-> `tengjiqi@gmail.com` directly.

8
README_zhCN.md
View File

@ -5,10 +5,10 @@
</p>
<p align="center">
<a href="https://github.com/KFCFans/PowerJob/actions"><img src="https://github.com/KFCFans/PowerJob/workflows/Java%20CI%20with%20Maven/badge.svg?branch=master" alt="actions"></a>
<a href="https://github.com/PowerJob/PowerJob/actions"><img src="https://github.com/PowerJob/PowerJob/workflows/Java%20CI%20with%20Maven/badge.svg?branch=master" alt="actions"></a>
<a href="https://search.maven.org/search?q=com.github.kfcfans"><img alt="Maven Central" src="https://img.shields.io/maven-central/v/com.github.kfcfans/powerjob-worker"></a>
<a href="https://github.com/KFCFans/PowerJob/releases"><img alt="GitHub release (latest SemVer)" src="https://img.shields.io/github/v/release/kfcfans/powerjob?color=%23E59866"></a>
<a href="https://github.com/KFCFans/PowerJob/blob/master/LICENSE"><img src="https://img.shields.io/github/license/KFCFans/PowerJob" alt="LICENSE"></a>
<a href="https://github.com/PowerJob/PowerJob/releases"><img alt="GitHub release (latest SemVer)" src="https://img.shields.io/github/v/release/kfcfans/powerjob?color=%23E59866"></a>
<a href="https://github.com/PowerJob/PowerJob/blob/master/LICENSE"><img src="https://img.shields.io/github/license/KFCFans/PowerJob" alt="LICENSE"></a>
</p>
PowerJob（原OhMyScheduler）是全新一代分布式调度与计算框架，能让您轻松完成作业的调度与繁杂任务的分布式计算。
@ -62,7 +62,7 @@ PowerJob 的设计目标为企业级的分布式任务调度平台，即成为
PS：感谢文档翻译平台[breword](https://www.breword.com/)对本项目英文文档翻译做出的巨大贡献！
# 接入登记
[点击进行接入登记，为 PowerJob 的发展贡献自己的力量！](https://github.com/KFCFans/PowerJob/issues/6)
[点击进行接入登记，为 PowerJob 的发展贡献自己的力量！](https://github.com/PowerJob/PowerJob/issues/6)
ღ( ´・ᴗ・\` )ღ 感谢以下接入用户的大力支持 ღ( ´・ᴗ・\` )ღ

16
pom.xml
View File

@ -19,8 +19,8 @@
</license>
</licenses>
<scm>
<url>https://github.com/KFCFans/PowerJob</url>
<connection>https://github.com/KFCFans/PowerJob.git</connection>
<url>https://github.com/PowerJob/PowerJob</url>
<connection>https://github.com/PowerJob/PowerJob.git</connection>
</scm>
<developers>
@ -87,7 +87,7 @@
<testTarget>${java.version}</testTarget>
</configuration>
</plugin>
<!-- 打包源码 -->
<!-- Package source codes -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
@ -121,9 +121,9 @@
<artifactId>maven-javadoc-plugin</artifactId>
<version>${maven-javadoc-plugin.version}</version>
<configuration>
<!-- JavaDoc 编译错误不影响正常构建 -->
<!-- Prevent JavaDoc error from affecting building project. -->
<failOnError>false</failOnError>
<!-- 非严格模式...以后要好好按格式写注释啊... -->
<!-- Non-strict mode -->
<additionalJOption>-Xdoclint:none</additionalJOption>
</configuration>
<executions>
@ -164,7 +164,7 @@
</distributionManagement>
</profile>
<!-- 本地使用 -->
<!-- Local profile -->
<profile>
<id>dev</id>
<activation>
@ -173,7 +173,7 @@
<build>
<plugins>
<!-- 编译插件 -->
<!-- Maven compiler plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
@ -185,7 +185,7 @@
<testTarget>${java.version}</testTarget>
</configuration>
</plugin>
<!-- 编辑 MANIFEST.MF -->
<!-- Edit MANIFEST.MF -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>

6
powerjob-client/pom.xml
View File

@ -10,13 +10,13 @@
<modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-client</artifactId>
<version>3.4.2</version>
<version>3.4.3</version>
<packaging>jar</packaging>
<properties>
<junit.version>5.6.1</junit.version>
<fastjson.version>1.2.68</fastjson.version>
<powerjob.common.version>3.4.2</powerjob.common.version>
<powerjob.common.version>3.4.3</powerjob.common.version>
<mvn.shade.plugin.version>3.2.4</mvn.shade.plugin.version>
</properties>
@ -37,7 +37,7 @@
<version>${powerjob.common.version}</version>
</dependency>
<!-- Junit 测试 -->
<!-- Junit tests -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>

209
powerjob-client/src/main/java/com/github/kfcfans/powerjob/client/OhMyClient.java
View File

@ -25,7 +25,7 @@ import java.util.Objects;
import static com.github.kfcfans.powerjob.client.TypeStore.*;
/**
* OpenAPI 客户端
* OhMyClient, the client for OpenAPI.
*
* @author tjq
* @since 2020/4/15
@ -40,9 +40,10 @@ public class OhMyClient {
private static final String URL_PATTERN = "http://%s%s%s";
/**
* 初始化 OhMyClient 客户端
* @param domain 比如 www.powerjob-server.com（内网域名，自行完成 DNS & Proxy）
* @param appName 负责的应用名称
* Init OhMyClient with domain, appName and password.
* @param domain like powerjob-server.apple-inc.com (Intranet Domain)
* @param appName name of the application
* @param password password of the application
*/
public OhMyClient(String domain, String appName, String password) {
this(Lists.newArrayList(domain), appName, password);
@ -50,9 +51,10 @@ public class OhMyClient {
/**
* 初始化 OhMyClient 客户端
* @param addressList IP:Port 列表
* @param appName 负责的应用名称
* Init OhMyClient with server address, appName and password.
* @param addressList IP:Port address list, like 192.168.1.1:7700
* @param appName name of the application
* @param password password of the application
*/
public OhMyClient(List<String> addressList, String appName, String password) {
@ -79,7 +81,7 @@ public class OhMyClient {
}
if (StringUtils.isEmpty(currentAddress)) {
throw new PowerJobException("no server available");
throw new PowerJobException("no server available for OhMyClient");
}
log.info("[OhMyClient] {}'s OhMyClient bootstrap successfully, using server: {}", appName, currentAddress);
}
@ -101,12 +103,12 @@ public class OhMyClient {
/* ************* Job 区 ************* */
/**
* 保存任务（包括创建与修改）
* @param request 任务详细参数
* @return 创建的任务ID
* @throws PowerJobException 异常
* Save one Job
* When an ID exists in SaveJobInfoRequest, it is an update operation. Otherwise, it is a crate operation.
* @param request Job meta info
* @return jobId
*/
public ResultDTO<Long> saveJob(SaveJobInfoRequest request) throws PowerJobException {
public ResultDTO<Long> saveJob(SaveJobInfoRequest request) {
request.setAppId(appId);
MediaType jsonType = MediaType.parse("application/json; charset=utf-8");
@ -116,12 +118,11 @@ public class OhMyClient {
}
/**
* 根据 jobId 查询任务信息
* @param jobId 任务ID
* @return 任务详细信息
* @throws PowerJobException 异常
* Query JobInfo by jobId
* @param jobId jobId
* @return Job meta info
*/
public ResultDTO<JobInfoDTO> fetchJob(Long jobId) throws PowerJobException {
public ResultDTO<JobInfoDTO> fetchJob(Long jobId) {
RequestBody body = new FormBody.Builder()
.add("jobId", jobId.toString())
.add("appId", appId.toString())
@ -131,12 +132,11 @@ public class OhMyClient {
}
/**
* 禁用某个任务
* @param jobId 任务ID
* @return 标准返回对象
* @throws PowerJobException 异常
* Disable one Job by jobId
* @param jobId jobId
* @return Standard return object
*/
public ResultDTO<Void> disableJob(Long jobId) throws PowerJobException {
public ResultDTO<Void> disableJob(Long jobId) {
RequestBody body = new FormBody.Builder()
.add("jobId", jobId.toString())
.add("appId", appId.toString())
@ -146,12 +146,11 @@ public class OhMyClient {
}
/**
* 启用某个任务
* @param jobId 任务ID
* @return 标准返回对象
* @throws PowerJobException 异常
* Enable one job by jobId
* @param jobId jobId
* @return Standard return object
*/
public ResultDTO<Void> enableJob(Long jobId) throws PowerJobException {
public ResultDTO<Void> enableJob(Long jobId) {
RequestBody body = new FormBody.Builder()
.add("jobId", jobId.toString())
.add("appId", appId.toString())
@ -161,12 +160,11 @@ public class OhMyClient {
}
/**
* 删除某个任务
* @param jobId 任务ID
* @return 标准返回对象
* @throws PowerJobException 异常
* Delete one job by jobId
* @param jobId jobId
* @return Standard return object
*/
public ResultDTO<Void> deleteJob(Long jobId) throws PowerJobException {
public ResultDTO<Void> deleteJob(Long jobId) {
RequestBody body = new FormBody.Builder()
.add("jobId", jobId.toString())
.add("appId", appId.toString())
@ -176,14 +174,13 @@ public class OhMyClient {
}
/**
* 运行某个任务
* @param jobId 任务ID
* @param instanceParams 任务实例的参数
* @param delayMS 延迟时间，单位毫秒
* @return 任务实例ID（instanceId）
* @throws PowerJobException 异常
* Run a job once
* @param jobId ID of the job to be run
* @param instanceParams Runtime parameters of the job (TaskContext#instanceParams)
* @param delayMS Delay time（Milliseconds）
* @return instanceId
*/
public ResultDTO<Long> runJob(Long jobId, String instanceParams, long delayMS) throws PowerJobException {
public ResultDTO<Long> runJob(Long jobId, String instanceParams, long delayMS) {
FormBody.Builder builder = new FormBody.Builder()
.add("jobId", jobId.toString())
.add("appId", appId.toString())
@ -195,18 +192,17 @@ public class OhMyClient {
String post = postHA(OpenAPIConstant.RUN_JOB, builder.build());
return JSONObject.parseObject(post, LONG_RESULT_TYPE);
}
public ResultDTO<Long> runJob(Long jobId) throws PowerJobException {
public ResultDTO<Long> runJob(Long jobId) {
return runJob(jobId, null, 0);
}
/* ************* Instance 区 ************* */
/* ************* Instance API list ************* */
/**
* 停止应用实例
* @param instanceId 应用实例ID
* @return true 停止成功，false 停止失败
* @throws PowerJobException 异常
* Stop one job instance
* @param instanceId instanceId
* @return Standard return object
*/
public ResultDTO<Void> stopInstance(Long instanceId) throws PowerJobException {
public ResultDTO<Void> stopInstance(Long instanceId) {
RequestBody body = new FormBody.Builder()
.add("instanceId", instanceId.toString())
.add("appId", appId.toString())
@ -216,13 +212,12 @@ public class OhMyClient {
}
/**
* 取消任务实例
* 接口使用条件：调用接口时间与待取消任务的预计执行时间有一定时间间隔，否则不保证可靠性！
* @param instanceId 任务实例ID
* @return true 代表取消成功，false 取消失败
* @throws PowerJobException 异常
* Cancel a job instance that is not yet running
* Notice：There is a time interval between the call interface time and the expected execution time of the job instance to be cancelled, otherwise reliability is not guaranteed
* @param instanceId instanceId
* @return Standard return object
*/
public ResultDTO<Void> cancelInstance(Long instanceId) throws PowerJobException {
public ResultDTO<Void> cancelInstance(Long instanceId) {
RequestBody body = new FormBody.Builder()
.add("instanceId", instanceId.toString())
.add("appId", appId.toString())
@ -232,13 +227,12 @@ public class OhMyClient {
}
/**
* 重试任务实例
* 只有完成状态（成功、失败、手动停止、被取消）的任务才能被重试，且暂不支持工作流内任务实例的重试
* @param instanceId 任务实例ID
* @return true 代表取消成功，false 取消失败
* @throws PowerJobException 异常
* Retry failed job instance
* Notice: Only job instance with completion status (success, failure, manually stopped, cancelled) can be retried, and retries of job instances within workflows are not supported yet.
* @param instanceId instanceId
* @return Standard return object
*/
public ResultDTO<Void> retryInstance(Long instanceId) throws PowerJobException {
public ResultDTO<Void> retryInstance(Long instanceId) {
RequestBody body = new FormBody.Builder()
.add("instanceId", instanceId.toString())
.add("appId", appId.toString())
@ -248,12 +242,11 @@ public class OhMyClient {
}
/**
* 查询任务实例状态
* @param instanceId 应用实例ID
* @return {@link InstanceStatus} 的枚举值
* @throws PowerJobException 异常
* Query status about a job instance
* @param instanceId instanceId
* @return {@link InstanceStatus}
*/
public ResultDTO<Integer> fetchInstanceStatus(Long instanceId) throws PowerJobException {
public ResultDTO<Integer> fetchInstanceStatus(Long instanceId) {
RequestBody body = new FormBody.Builder()
.add("instanceId", instanceId.toString())
.build();
@ -262,12 +255,11 @@ public class OhMyClient {
}
/**
* 查询任务实例的信息
* @param instanceId 任务实例ID
* @return 任务实例信息
* @throws PowerJobException 潜在的异常
* Query detail about a job instance
* @param instanceId instanceId
* @return instance detail
*/
public ResultDTO<InstanceInfoDTO> fetchInstanceInfo(Long instanceId) throws PowerJobException {
public ResultDTO<InstanceInfoDTO> fetchInstanceInfo(Long instanceId) {
RequestBody body = new FormBody.Builder()
.add("instanceId", instanceId.toString())
.build();
@ -275,14 +267,14 @@ public class OhMyClient {
return JSONObject.parseObject(post, INSTANCE_RESULT_TYPE);
}
/* ************* Workflow 区 ************* */
/* ************* Workflow API list ************* */
/**
* 保存工作流（包括创建和修改）
* @param request 创建/修改 Workflow 请求
* @return 工作流ID
* @throws PowerJobException 异常
* Save one workflow
* When an ID exists in SaveWorkflowRequest, it is an update operation. Otherwise, it is a crate operation.
* @param request Workflow meta info
* @return workflowId
*/
public ResultDTO<Long> saveWorkflow(SaveWorkflowRequest request) throws PowerJobException {
public ResultDTO<Long> saveWorkflow(SaveWorkflowRequest request) {
request.setAppId(appId);
MediaType jsonType = MediaType.parse(OmsConstant.JSON_MEDIA_TYPE);
// 中坑记录：用 FastJSON 序列化会导致 Server 接收时 pEWorkflowDAG 为 null，无语.jpg
@ -292,12 +284,11 @@ public class OhMyClient {
}
/**
* 根据 workflowId 查询工作流信息
* Query Workflow by workflowId
* @param workflowId workflowId
* @return 工作流信息
* @throws PowerJobException 异常
* @return Workflow meta info
*/
public ResultDTO<WorkflowInfoDTO> fetchWorkflow(Long workflowId) throws PowerJobException {
public ResultDTO<WorkflowInfoDTO> fetchWorkflow(Long workflowId) {
RequestBody body = new FormBody.Builder()
.add("workflowId", workflowId.toString())
.add("appId", appId.toString())
@ -307,12 +298,11 @@ public class OhMyClient {
}
/**
* 禁用某个工作流
* @param workflowId 工作流ID
* @return 标准返回对象
* @throws PowerJobException 异常
* Disable Workflow by workflowId
* @param workflowId workflowId
* @return Standard return object
*/
public ResultDTO<Void> disableWorkflow(Long workflowId) throws PowerJobException {
public ResultDTO<Void> disableWorkflow(Long workflowId) {
RequestBody body = new FormBody.Builder()
.add("workflowId", workflowId.toString())
.add("appId", appId.toString())
@ -322,12 +312,11 @@ public class OhMyClient {
}
/**
* 启用某个工作流
* Enable Workflow by workflowId
* @param workflowId workflowId
* @return 标准返回对象
* @throws PowerJobException 异常
* @return Standard return object
*/
public ResultDTO<Void> enableWorkflow(Long workflowId) throws PowerJobException {
public ResultDTO<Void> enableWorkflow(Long workflowId) {
RequestBody body = new FormBody.Builder()
.add("workflowId", workflowId.toString())
.add("appId", appId.toString())
@ -337,12 +326,11 @@ public class OhMyClient {
}
/**
* 删除某个工作流
* Delete Workflow by workflowId
* @param workflowId workflowId
* @return 标准返回对象
* @throws PowerJobException 异常
* @return Standard return object
*/
public ResultDTO<Void> deleteWorkflow(Long workflowId) throws PowerJobException {
public ResultDTO<Void> deleteWorkflow(Long workflowId) {
RequestBody body = new FormBody.Builder()
.add("workflowId", workflowId.toString())
.add("appId", appId.toString())
@ -352,14 +340,13 @@ public class OhMyClient {
}
/**
* 运行工作流
* @param workflowId 工作流ID
* @param initParams 启动参数
* @param delayMS 延迟时间，单位毫秒 ms
* @return 工作流实例ID
* @throws PowerJobException 异常信息
* Run a workflow once
* @param workflowId workflowId
* @param initParams workflow startup parameters
* @param delayMS Delay time（Milliseconds）
* @return workflow instanceId
*/
public ResultDTO<Long> runWorkflow(Long workflowId, String initParams, long delayMS) throws PowerJobException {
public ResultDTO<Long> runWorkflow(Long workflowId, String initParams, long delayMS) {
FormBody.Builder builder = new FormBody.Builder()
.add("workflowId", workflowId.toString())
.add("appId", appId.toString())
@ -370,18 +357,17 @@ public class OhMyClient {
String post = postHA(OpenAPIConstant.RUN_WORKFLOW, builder.build());
return JSONObject.parseObject(post, LONG_RESULT_TYPE);
}
public ResultDTO<Long> runWorkflow(Long workflowId) throws PowerJobException {
public ResultDTO<Long> runWorkflow(Long workflowId) {
return runWorkflow(workflowId, null, 0);
}
/* ************* Workflow Instance 区 ************* */
/* ************* Workflow Instance API list ************* */
/**
* 停止应用实例
* @param wfInstanceId 工作流实例ID
* @return true 停止成功 ； false 停止失败
* @throws PowerJobException 异常
* Stop one workflow instance
* @param wfInstanceId workflow instanceId
* @return Standard return object
*/
public ResultDTO<Void> stopWorkflowInstance(Long wfInstanceId) throws PowerJobException {
public ResultDTO<Void> stopWorkflowInstance(Long wfInstanceId) {
RequestBody body = new FormBody.Builder()
.add("wfInstanceId", wfInstanceId.toString())
.add("appId", appId.toString())
@ -391,12 +377,11 @@ public class OhMyClient {
}
/**
* 查询任务实例的信息
* @param wfInstanceId 任务实例ID
* @return 任务实例信息
* @throws PowerJobException 潜在的异常
* Query detail about a workflow instance
* @param wfInstanceId workflow instanceId
* @return detail about a workflow
*/
public ResultDTO<WorkflowInstanceInfoDTO> fetchWorkflowInstanceInfo(Long wfInstanceId) throws PowerJobException {
public ResultDTO<WorkflowInstanceInfoDTO> fetchWorkflowInstanceInfo(Long wfInstanceId) {
RequestBody body = new FormBody.Builder()
.add("wfInstanceId", wfInstanceId.toString())
.add("appId", appId.toString())
@ -439,6 +424,6 @@ public class OhMyClient {
}
log.error("[OhMyClient] do post for path: {} failed because of no server available in {}.", path, allAddress);
throw new PowerJobException("no server available when send post");
throw new PowerJobException("no server available when send post request");
}
}

2
powerjob-client/src/main/java/com/github/kfcfans/powerjob/client/TypeStore.java
View File

@ -4,7 +4,7 @@ import com.alibaba.fastjson.TypeReference;
import com.github.kfcfans.powerjob.common.response.*;
/**
* 类型工厂
* TypeReference store.
*
* @author tjq
* @since 11/7/20

4
powerjob-client/src/test/java/TestClient.java
View File

@ -12,7 +12,7 @@ import org.junit.jupiter.api.Test;
import java.util.concurrent.TimeUnit;
/**
* 测试 Client
* Test cases for {@link OhMyClient}
*
* @author tjq
* @since 2020/4/15
@ -110,7 +110,7 @@ public class TestClient {
ResultDTO<Long> startRes = ohMyClient.runJob(15L, "start by OhMyClient", 2000000);
System.out.println("runJob result: " + JSONObject.toJSONString(startRes));
// 手动重启 server，干掉时间轮中的调度数据
// Restart server manually and clear all the data in time wheeler.
TimeUnit.MINUTES.sleep(1);
ResultDTO<Void> cancelRes = ohMyClient.cancelInstance(startRes.getData());

2
powerjob-client/src/test/java/TestWorkflow.java
View File

@ -13,7 +13,7 @@ import org.junit.jupiter.api.Test;
import java.util.List;
/**
* 测试 Client（workflow部分）
* Test cases for {@link OhMyClient} workflow.
*
* @author tjq
* @since 2020/6/2

4
powerjob-common/pom.xml
View File

@ -10,7 +10,7 @@
<modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-common</artifactId>
<version>3.4.2</version>
<version>3.4.3</version>
<packaging>jar</packaging>
<properties>
@ -76,7 +76,7 @@
<version>${commons.io.version}</version>
</dependency>
<!-- Junit 测试 -->
<!-- Junit tests -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>

9
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/ContainerConstant.java
View File

@ -1,7 +1,7 @@
package com.github.kfcfans.powerjob.common;
/**
* 容器常量
* Container constants.
*
* @author tjq
* @since 2020/5/15
@ -9,13 +9,16 @@ package com.github.kfcfans.powerjob.common;
public class ContainerConstant {
/**
* spring-context 配置文件名称
* Spring-context configuration file name of the container.
*/
public static final String SPRING_CONTEXT_FILE_NAME = "oms-worker-container-spring-context.xml";
/**
* container 属性文件名称
* Property file name of the container.
*/
public static final String CONTAINER_PROPERTIES_FILE_NAME = "oms-worker-container.properties";
/**
* Package name of the container.
*/
public static final String CONTAINER_PACKAGE_NAME_KEY = "PACKAGE_NAME";
}

11
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/Env.java
View File

@ -1,13 +1,22 @@
package com.github.kfcfans.powerjob.common;
/**
* 部署环境
* Environment Enum class.
*
* @author tjq
* @since 2020/5/3
*/
public enum Env {
/**
* Development or test environment.
*/
DAILY,
/**
* Pre-release environment.
*/
PRE,
/**
* Production environment.
*/
PRODUCT
}

11
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/ExecuteType.java
View File

@ -4,7 +4,7 @@ import lombok.AllArgsConstructor;
import lombok.Getter;
/**
* 任务执行类型
* Execution type.
*
* @author tjq
* @since 2020/3/17
@ -12,8 +12,17 @@ import lombok.Getter;
@Getter
@AllArgsConstructor
public enum ExecuteType {
/**
* Standalone type of task.
*/
STANDALONE(1, "单机执行"),
/**
* Broadcast type of task.
*/
BROADCAST(2, "广播执行"),
/**
* MapReduce type of task.
*/
MAP_REDUCE(3, "MapReduce"),
MAP(4, "Map");

2
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/InstanceStatus.java
View File

@ -7,7 +7,7 @@ import lombok.Getter;
import java.util.List;
/**
* 任务运行状态
* Status of the job instance
*
* @author tjq
* @since 2020/3/17

2
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/OmsConstant.java
View File

@ -1,7 +1,7 @@
package com.github.kfcfans.powerjob.common;
/**
* 公共常量
* Common constants.
*
* @author tjq
* @since 2020/5/31

2
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/OmsSerializable.java
View File

@ -3,7 +3,7 @@ package com.github.kfcfans.powerjob.common;
import java.io.Serializable;
/**
* PowerJob 序列化标记接口
* PowerJob serializable interface.
*
* @author tjq
* @since 2020/4/16

6
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/ProcessorType.java
View File

@ -4,7 +4,7 @@ import lombok.AllArgsConstructor;
import lombok.Getter;
/**
* 处理器类型
* Task Processor Type
*
* @author tjq
* @since 2020/3/23
@ -18,8 +18,8 @@ public enum ProcessorType {
PYTHON(3, "Python脚本"),
JAVA_CONTAINER(4, "Java容器");
private int v;
private String des;
private final int v;
private final String des;
public static ProcessorType of(int v) {
for (ProcessorType type : values()) {

8
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/TimeExpressionType.java
View File

@ -7,7 +7,7 @@ import lombok.Getter;
import java.util.List;
/**
* 时间表达式类型
* Scheduling time strategies
*
* @author tjq
* @since 2020/3/30
@ -18,13 +18,13 @@ public enum TimeExpressionType {
API(1),
CRON(2),
FIX_RATE(3),
FIX_DELAY(4),
FIXED_RATE(3),
FIXED_DELAY(4),
WORKFLOW(5);
int v;
public static final List<Integer> frequentTypes = Lists.newArrayList(FIX_RATE.v, FIX_DELAY.v);
public static final List<Integer> frequentTypes = Lists.newArrayList(FIXED_RATE.v, FIXED_DELAY.v);
public static TimeExpressionType of(int v) {
for (TimeExpressionType type : values()) {

18
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/model/DeployedContainerInfo.java
View File

@ -6,7 +6,7 @@ import lombok.Data;
import lombok.NoArgsConstructor;
/**
* 已部署的容器信息
* Deployed Container Information
*
* @author tjq
* @since 2020/5/18
@ -16,12 +16,20 @@ import lombok.NoArgsConstructor;
@AllArgsConstructor
public class DeployedContainerInfo implements OmsSerializable {
// 容器ID
/**
* Id of the container.
*/
private Long containerId;
// 版本
/**
* Version of the container.
*/
private String version;
// 部署时间
/**
* Deploy timestamp.
*/
private long deployedTime;
// 机器地址（无需上报）
/**
* No need to report to the server
*/
private String workerAddress;
}

18
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/model/GitRepoInfo.java
View File

@ -3,19 +3,27 @@ package com.github.kfcfans.powerjob.common.model;
import lombok.Data;
/**
* Git代码库信息
* The class for Git Repository info.
*
* @author tjq
* @since 2020/5/17
*/
@Data
public class GitRepoInfo {
// 仓库地址
/**
* Address of Git repository.
*/
private String repo;
// 分支名称
/**
* Name of the branch.
*/
private String branch;
// 用户名
/**
* username of Git.
*/
private String username;
// 密码
/**
* Password of Git.
*/
private String password;
}

56
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/model/InstanceDetail.java
View File

@ -7,7 +7,7 @@ import lombok.NoArgsConstructor;
import java.util.List;
/**
* 任务实例的运行详细信息
* Detailed info of job instances.
*
* @author tjq
* @since 2020/4/11
@ -16,33 +16,59 @@ import java.util.List;
@NoArgsConstructor
public class InstanceDetail implements OmsSerializable {
// 任务预计执行时间
/**
* Expected trigger time.
*/
private Long expectedTriggerTime;
// 任务整体开始时间
/**
* Actual trigger time of an instance.
*/
private Long actualTriggerTime;
// 任务整体结束时间（可能不存在）
/**
* Finish time of an instance, which may be null.
*/
private Long finishedTime;
// 任务状态
/**
* Status of the task instance.
*/
private Integer status;
// 任务执行结果（可能不存在）
/**
* Execution result, which may be null.
*/
private String result;
// TaskTracker地址
/**
* Task tracker address.
*/
private String taskTrackerAddress;
// 启动参数
/**
* Param string that is passed to an instance when it is initialized.
*/
private String instanceParams;
// MR或BD任务专用
/**
* Task detail, used by MapReduce or Broadcast tasks.
*/
private TaskDetail taskDetail;
// 秒级任务专用
/**
* Sub instance details, used by frequent tasks.
*/
private List<SubInstanceDetail> subInstanceDetails;
// 重试次数
/**
* Running times.
*/
private Long runningTimes;
// 扩展字段，中间件升级不易，最好不要再改 common 包了...否则 server worker 版本不兼容
/**
* Extended fields. Middlewares are not supposed to update frequently.
* Changes in PowerJob-common may lead to incompatible versions.
* PowerJob-common packages should not be modified if not necessary.
*/
private String extra;
// 秒级任务的 extra -> List<SubInstanceDetail>
/**
* Extra info for frequent tasks, return List<SubInstanceDetail>.
*/
@Data
@NoArgsConstructor
public static class SubInstanceDetail implements OmsSerializable {
@ -53,7 +79,9 @@ public class InstanceDetail implements OmsSerializable {
private int status;
}
// MapReduce 和 Broadcast 任务的 extra ->
/**
* Extra info of {@code MapReduce} or {@code Broadcast} type of tasks.
*/
@Data
@NoArgsConstructor
public static class TaskDetail implements OmsSerializable {

18
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/model/InstanceLogContent.java
View File

@ -6,7 +6,7 @@ import lombok.Data;
import lombok.NoArgsConstructor;
/**
* 任务实例日志对象
* Log instance model.
*
* @author tjq
* @since 2020/4/21
@ -16,13 +16,21 @@ import lombok.NoArgsConstructor;
@AllArgsConstructor
public class InstanceLogContent implements OmsSerializable {
// 实例ID
/**
* Id of instance.
*/
private long instanceId;
// 日志提交时间
/**
* Submitted time of the log.
*/
private long logTime;
// 级别
/**
* Level of the log.
*/
private int logLevel;
// 日志内容
/**
* Content of the log.
*/
private String logContent;
}

22
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/model/PEWorkflowDAG.java
View File

@ -13,8 +13,7 @@ import java.io.Serializable;
import java.util.List;
/**
* Point & Edge DAG 表示法
* 点 + 线，易于表达和传播
* Points & edges for DAG, making it easier to describe or transfer.
*
* @author tjq
* @since 2020/5/26
@ -23,11 +22,18 @@ import java.util.List;
@NoArgsConstructor
public class PEWorkflowDAG implements Serializable {
// DAG 图（点线表示法）
/**
* Nodes of DAG diagram.
*/
private List<Node> nodes;
/**
* Edges of DAG diagram.
*/
private List<Edge> edges;
// 点
/**
* Point.
*/
@Data
@NoArgsConstructor
@AllArgsConstructor
@ -35,7 +41,9 @@ public class PEWorkflowDAG implements Serializable {
private Long jobId;
private String jobName;
// 运行时参数，图定义不需要
/**
* Instance running param, which is not required by DAG.
*/
@JsonSerialize(using= ToStringSerializer.class)
private Long instanceId;
private Integer status;
@ -47,7 +55,9 @@ public class PEWorkflowDAG implements Serializable {
}
}
// 边 jobId -> jobId
/**
* Edge formed by two job ids.
*/
@Data
@NoArgsConstructor
@AllArgsConstructor

73
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/model/SystemMetrics.java
View File

@ -4,7 +4,7 @@ import com.github.kfcfans.powerjob.common.OmsSerializable;
import lombok.Data;
/**
* 系统指标
* Class for system metrics.
*
* @author tjq
* @since 2020/3/25
@ -12,61 +12,87 @@ import lombok.Data;
@Data
public class SystemMetrics implements OmsSerializable, Comparable<SystemMetrics> {
// CPU核心数量
/**
* CPU processor num.
*/
private int cpuProcessors;
// CPU负载（负载 和 使用率 是两个完全不同的概念，Java 无法获取 CPU 使用率，只能获取负载）
/**
* Percent of CPU load.
*/
private double cpuLoad;
// 内存（单位 GB）
/**
* Memory that is used by JVM, in GB.
*/
private double jvmUsedMemory;
/**
* Max memory that JVM can use, in GB.
*/
private double jvmMaxMemory;
// 内存占用（0.X，非百分比）
/**
* Ratio of memory that JVM uses to total memory, 0.X,
* the value is between 0 and 1.
*/
private double jvmMemoryUsage;
// 磁盘（单位 GB）
/**
* Total used disk space, in GB.
*/
private double diskUsed;
/**
* Total disk space, in GB.
*/
private double diskTotal;
// 磁盘占用（0.X，非百分比）
/**
* Used disk ratio.
*/
private double diskUsage;
// 缓存分数
/**
* Score of cache.
*/
private int score;
/**
* Override compareTo.
*
* @param that the metrics that is to be compared with current.
* @return {@code int}
*/
@Override
public int compareTo(SystemMetrics that) {
// 降序排列
// Sort by metrics in descending order.
return that.calculateScore() - this.calculateScore();
}
/**
* 计算得分情况，内存 & CPU (磁盘不参与计算)
* @return 得分情况
* Calculate score, based on CPU and memory info.
*
* @return score
*/
public int calculateScore() {
if (score > 0) {
return score;
}
// 对于 TaskTracker 来说，内存是任务顺利完成的关键，因此内存 2 块钱 1GB
// Memory is vital to TaskTracker, so we set the multiplier factor as 2.
double memScore = (jvmMaxMemory - jvmUsedMemory) * 2;
// CPU 剩余负载，1 块钱 1 斤
// Calculate the remaining load of CPU. Multiplier is set as 1.
double cpuScore = cpuProcessors - cpuLoad;
// Indian Windows 无法获取 CpuLoad，为 -1，固定为 1
// Windows can not fetch CPU load, set cpuScore as 1.
if (cpuScore > cpuProcessors) {
cpuScore = 1;
}
score = (int) (memScore + cpuScore);
return score;
}
/**
* 该机器是否可用
* @param minCPUCores 判断标准之最低可用CPU核心数量
* @param minMemorySpace 判断标准之最低可用内存
* @param minDiskSpace 判断标准之最低可用磁盘空间
* @return 是否可用
* Judge if the machine is available.
*
* @param minCPUCores Minimum available CPU cores.
* @param minMemorySpace Minimum available memory size
* @param minDiskSpace Minimum disk space
* @return {@code boolean} whether the machine is available.
*/
public boolean available(double minCPUCores, double minMemorySpace, double minDiskSpace) {
@ -77,7 +103,8 @@ public class SystemMetrics implements OmsSerializable, Comparable<SystemMetrics>
return false;
}
// cpuLoad 为负数代表无法获取，不判断。等于 0 为最理想情况，CPU 空载，不需要判断
// 0 indicates the CPU is free, which is the optimal condition.
// Negative number means being unable to fetch CPU info, return true.
if (cpuLoad <= 0 || minCPUCores <= 0) {
return true;
}

106
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/request/http/SaveJobInfoRequest.java
View File

@ -9,7 +9,7 @@ import lombok.Data;
import java.util.List;
/**
* 创建/修改 JobInfo 请求
* Save or modify {@link com.github.kfcfans.powerjob.common.response.JobInfoDTO}
*
* @author tjq
* @since 2020/3/30
@ -17,68 +17,118 @@ import java.util.List;
@Data
public class SaveJobInfoRequest {
// 任务ID（jobId），null -> 插入，否则为更新
/**
* id of the job. set null to create or non-null to update the job.
*/
private Long id;
/* ************************** 任务基本信息 ************************** */
// 任务名称
/* ************************** Base info of related job. ************************** */
/**
* Name of the job.
*/
private String jobName;
// 任务描述
/**
* Description of the job.
*/
private String jobDescription;
// 任务所属的应用ID（Client无需填写该参数，自动填充）
/**
* Related id of the application. There is not need to set this property
* in PowerJob-client, as it would be set automatically.
*/
private Long appId;
// 任务自带的参数
/**
* Params that these jobs carry with when they are created.
*/
private String jobParams;
/* ************************** 定时参数 ************************** */
// 时间表达式类型（CRON/API/FIX_RATE/FIX_DELAY）
/* ************************** Timing param. ************************** */
/**
* Time expression type.
*/
private TimeExpressionType timeExpressionType;
// 时间表达式，CRON/NULL/LONG/LONG
/**
* Time expression.
*/
private String timeExpression;
/* ************************** 执行方式 ************************** */
// 执行类型，单机/广播/MR
/* ************************** Execution type. ************************** */
/**
* Execution type, {@code standalone}, {@code broadcast} or {@code Map/MapReduce}
*/
private ExecuteType executeType;
// 执行器类型，Java/Shell
/**
* Processor type, {@code Java}, {@code Python} or {@code Shell}.
*/
private ProcessorType processorType;
// 执行器信息
/**
* Processor info.
*/
private String processorInfo;
/* ************************** 运行时配置 ************************** */
// 最大同时运行任务数，0 代表不限
/* ************************** Running instance setting. ************************** */
/**
* Maximum instance setting num. {@code 0} means there is no restriction.
*/
private Integer maxInstanceNum = 0;
// 并发度，同时执行的线程数量
/**
* Concurrency setting. Number of threads that run simultaneously.
*/
private Integer concurrency = 5;
// 任务整体超时时间
/**
* Max instance running time setting. {@code 0L} means there is no restriction.
*/
private Long instanceTimeLimit = 0L;
/* ************************** 重试配置 ************************** */
/* ************************** Retrial setting. ************************** */
/**
* Instance retry number setting.
*/
private Integer instanceRetryNum = 0;
/**
* Task retry number setting.
*/
private Integer taskRetryNum = 0;
/* ************************** 繁忙机器配置 ************************** */
// 最低CPU核心数量，0代表不限
/* ************************** Busy Machine setting. ************************** */
/**
* Minimum CPU required. {@code 0} means there is no restriction.
*/
private double minCpuCores = 0;
// 最低内存空间，单位 GB，0代表不限
/**
* Minimum memory required, in GB.
*/
private double minMemorySpace = 0;
// 最低磁盘空间，单位 GB，0代表不限
/**
* Minimum disk space, in GB. {@code 0} means there is no restriction.
*/
private double minDiskSpace = 0;
// 1 正常运行，2 停止（不再调度）
private boolean enable = true;
/* ************************** 集群配置 ************************** */
// 指定机器运行，空代表不限，非空则只会使用其中的机器运行（多值逗号分割）
/* ************************** PowerJob-worker cluster property ************************** */
/**
* Designated PowerJob-worker nodes. Blank value indicates that there is
* no limit. Non-blank value means to run the corresponding machine(s) only.
* example: 192.168.1.1:27777,192.168.1.2:27777
*/
private String designatedWorkers;
// 最大机器数量
/**
* Max count of PowerJob-worker nodes.
*/
private Integer maxWorkerCount = 0;
// 报警用户ID列表
/**
* The id list of the users that need to be notified.
*/
private List<Long> notifyUserIds;
/**
* Check non-null properties.
*/
public void valid() {
CommonUtils.requireNonNull(jobName, "jobName can't be empty");
CommonUtils.requireNonNull(appId, "appId can't be empty");

2
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/response/InstanceInfoDTO.java
View File

@ -6,7 +6,7 @@ import lombok.Data;
import java.util.Date;
/**
* instanceInfo 对外输出对象
* instanceInfo Network transmission object
*
* @author tjq
* @since 2020/5/14

4
powerjob-common/src/main/java/com/github/kfcfans/powerjob/common/response/ResultDTO.java
View File

@ -7,7 +7,7 @@ import lombok.ToString;
import org.apache.commons.lang3.exception.ExceptionUtils;
/**
* 请求返回的结果对象
* The result object returned by the request
*
* @author tjq
* @since 2020/3/30
@ -18,9 +18,7 @@ import org.apache.commons.lang3.exception.ExceptionUtils;
public class ResultDTO<T> implements OmsSerializable {
private boolean success;
// 数据（success为 true 时存在）
private T data;
// 错误信息（success为 false 时存在）
private String message;
public static <T> ResultDTO<T> success(T data) {

25
powerjob-server/pom.xml
View File

@ -10,14 +10,14 @@
<modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-server</artifactId>
<version>3.4.2</version>
<version>3.4.3</version>
<packaging>jar</packaging>
<properties>
<swagger.version>2.9.2</swagger.version>
<springboot.version>2.3.4.RELEASE</springboot.version>
<powerjob.common.version>3.4.2</powerjob.common.version>
<!-- 数据库驱动版本，使用的是spring-boot-dependencies管理的版本 -->
<powerjob.common.version>3.4.3</powerjob.common.version>
<!-- MySQL version that corresponds to spring-boot-dependencies version. -->
<mysql.version>8.0.19</mysql.version>
<ojdbc.version>19.7.0.0</ojdbc.version>
<mssql-jdbc.version>7.4.1.jre8</mssql-jdbc.version>
@ -32,7 +32,7 @@
<fastjson.version>1.2.68</fastjson.version>
<dingding.version>1.0.1</dingding.version>
<!-- 部署时跳过该module -->
<!-- Skip this module when deploying. -->
<maven.deploy.skip>true</maven.deploy.skip>
</properties>
@ -136,42 +136,42 @@
<scope>test</scope>
</dependency>
<!-- zip4j（Zip操作） -->
<!-- zip4j(Zip support) -->
<dependency>
<groupId>net.lingala.zip4j</groupId>
<artifactId>zip4j</artifactId>
<version>${zip4j.version}</version>
</dependency>
<!-- jGit（Git操作） -->
<!-- jGit(Git support) -->
<dependency>
<groupId>org.eclipse.jgit</groupId>
<artifactId>org.eclipse.jgit</artifactId>
<version>${jgit.version}</version>
</dependency>
<!-- 时间工具类，NTP时间同步 -->
<!-- Time tools(NTP support) -->
<dependency>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<version>${commons.net.version}</version>
</dependency>
<!-- Maven Invoker（编译 maven 项目） -->
<!-- Maven Invoker(used for maven compilation) -->
<dependency>
<groupId>org.apache.maven.shared</groupId>
<artifactId>maven-invoker</artifactId>
<version>${mvn.invoker.version}</version>
</dependency>
<!-- fastJson（为了序列化 DAG 引用引入） -->
<!-- fastJson(used for serialization of DAG) -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>${fastjson.version}</version>
</dependency>
<!-- 钉钉 报警通知 -->
<!-- DingTalk SDK. -->
<dependency>
<groupId>com.aliyun</groupId>
<artifactId>alibaba-dingtalk-service-sdk</artifactId>
@ -185,7 +185,6 @@
</dependency>
<!-- swagger2 -->
<dependency>
<groupId>io.springfox</groupId>
@ -201,7 +200,7 @@
</dependencies>
<!-- SpringBoot专用的打包插件 -->
<!-- SpringBoot maven plugin -->
<build>
<plugins>
<plugin>
@ -215,7 +214,7 @@
<execution>
<goals>
<goal>build-info</goal>
<goal>repackage</goal><!--可以把依赖的包都打包到生成的Jar包中-->
<goal>repackage</goal>
</goals>
</execution>
</executions>

9
powerjob-server/src/main/java/com/github/kfcfans/powerjob/server/OhMyApplication.java
View File

@ -7,7 +7,7 @@ import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableScheduling;
/**
* SpringBoot 启动入口
* powerjob-server entry
*
* @author tjq
* @since 2020/3/29
@ -26,16 +26,15 @@ public class OhMyApplication {
public static void main(String[] args) {
// 完成前置工作
pre();
// 先启动 ActorSystem
// Init ActorSystem first
OhMyServer.init();
// 再启动SpringBoot
// Start SpringBoot application.
try {
SpringApplication.run(OhMyApplication.class, args);
}catch (Throwable t) {
} catch (Throwable t) {
log.error(TIPS);
throw t;
}

2
powerjob-server/src/main/java/com/github/kfcfans/powerjob/server/common/config/SwaggerConfig.java
View File

@ -44,7 +44,7 @@ public class SwaggerConfig {
.title("PowerJob")
.description("Distributed scheduling and computing framework.")
.license("Apache Licence 2")
.termsOfServiceUrl("https://github.com/KFCFans/PowerJob")
.termsOfServiceUrl("https://github.com/PowerJob/PowerJob")
.version(version)
.build();

4
powerjob-server/src/main/java/com/github/kfcfans/powerjob/server/service/ValidateService.java
View File

@ -33,8 +33,8 @@ public class ValidateService {
case API: return Lists.newArrayList(OmsConstant.NONE);
case WORKFLOW: return Lists.newArrayList("VALID: depends on workflow");
case CRON: return calculateCronExpression(timeExpression);
case FIX_RATE: return calculateFixRate(timeExpression);
case FIX_DELAY: return Lists.newArrayList("VALID: depends on execution cost time");
case FIXED_RATE: return calculateFixRate(timeExpression);
case FIXED_DELAY: return Lists.newArrayList("VALID: depends on execution cost time");
}
// impossible
return Collections.emptyList();

18
powerjob-server/src/main/resources/application-daily.properties
View File

@ -1,7 +1,7 @@
oms.env=DAILY
logging.config=classpath:logback-dev.xml
####### 外部数据库配置（需要用户更改为自己的数据库配置） #######
####### Database properties(Configure according to the the environment) #######
spring.datasource.core.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.core.jdbc-url=jdbc:mysql://localhost:3306/powerjob-daily?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
spring.datasource.core.username=root
@ -9,11 +9,13 @@ spring.datasource.core.password=No1Bug2Please3!
spring.datasource.core.hikari.maximum-pool-size=20
spring.datasource.core.hikari.minimum-idle=5
####### mongoDB配置，非核心依赖，通过配置 oms.mongodb.enable=false 来关闭 #######
####### MongoDB properties(Non-core configuration properties) #######
####### configure oms.mongodb.enable=false to disable mongodb #######
oms.mongodb.enable=true
spring.data.mongodb.uri=mongodb+srv://zqq:No1Bug2Please3!@cluster0.wie54.gcp.mongodb.net/powerjob_daily?retryWrites=true&w=majority
####### 邮件配置（不需要邮件报警可以删除以下配置来避免报错） #######
####### Email properties(Non-core configuration properties) #######
####### Delete the following code to disable the mail #######
spring.mail.host=smtp.163.com
spring.mail.username=zqq@163.com
spring.mail.password=GOFZPNARMVKCGONV
@ -21,18 +23,20 @@ spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true
####### 钉钉报警配置（不需要钉钉报警可以删除以下配置来避免报错） #######
####### DingTalk properties(Non-core configuration properties) #######
####### Delete the following code to disable the DingTalk #######
oms.alarm.ding.app-key=dingauqwkvxxnqskknfv
oms.alarm.ding.app-secret=XWrEPdAZMPgJeFtHuL0LH73LRj-74umF2_0BFcoXMfvnX0pCQvt0rpb1JOJU_HLl
oms.alarm.ding.agent-id=847044348
####### 资源清理配置 #######
####### Resource cleaning properties #######
oms.instanceinfo.retention=1
oms.container.retention.local=1
oms.container.retention.remote=-1
####### 缓存配置 #######
####### Cache properties #######
oms.instance.metadata.cache.size=1024
####### 精确获取 server 的百分比，0～100，100代表每次 worker 获取 server 都会进行完整的探活流程，不存在脑裂问题，但有性能开销 #######
####### Threshold in precise fetching server(0~100). 100 means full detection of server, in which #######
####### split-brain could be avoided while performance overhead would increase. #######
oms.accurate.select.server.percentage = 50

18
powerjob-server/src/main/resources/application-pre.properties
View File

@ -1,7 +1,7 @@
oms.env=PRE
logging.config=classpath:logback-product.xml
####### 数据库配置 #######
####### Database properties(Configure according to the the environment) #######
spring.datasource.core.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.core.jdbc-url=jdbc:mysql://remotehost:3306/powerjob-pre?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
spring.datasource.core.username=root
@ -9,11 +9,13 @@ spring.datasource.core.password=No1Bug2Please3!
spring.datasource.core.hikari.maximum-pool-size=20
spring.datasource.core.hikari.minimum-idle=5
####### mongoDB配置，非核心依赖，通过配置 oms.mongodb.enable=false 来关闭 #######
####### MongoDB properties(Non-core configuration properties) #######
####### configure oms.mongodb.enable=false to disable mongodb #######
oms.mongodb.enable=true
spring.data.mongodb.uri=mongodb://remotehost:27017/powerjob-pre
####### 邮件配置（不需要邮件报警可以删除以下配置来避免报错） #######
####### Email properties(Non-core configuration properties) #######
####### Delete the following code to disable the mail #######
spring.mail.host=smtp.qq.com
spring.mail.username=zqq
spring.mail.password=qqz
@ -21,18 +23,20 @@ spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true
####### 钉钉报警配置（不需要钉钉报警可以删除以下配置来避免报错） #######
####### DingTalk properties(Non-core configuration properties) #######
####### Delete the following code to disable the DingTalk #######
oms.alarm.ding.app-key=dingauqwkvxxnqskknfv
oms.alarm.ding.app-secret=XWrEPdAZMPgJeFtHuL0LH73LRj-74umF2_0BFcoXMfvnX0pCQvt0rpb1JOJU_HLl
oms.alarm.ding.agent-id=847044348
####### 资源清理配置 #######
####### Resource cleaning properties #######
oms.instanceinfo.retention=3
oms.container.retention.local=3
oms.container.retention.remote=-1
####### 缓存配置 #######
####### Cache properties #######
oms.instance.metadata.cache.size=1024
####### 精确获取 server 的百分比，0～100，100代表每次 worker 获取 server 都会进行完整的探活流程，不存在脑裂问题，但有性能开销 #######
####### Threshold in precise fetching server(0~100). 100 means full detection of server, in which #######
####### split-brain could be avoided while performance overhead would increase. #######
oms.accurate.select.server.percentage = 50

18
powerjob-server/src/main/resources/application-product.properties
View File

@ -1,7 +1,7 @@
oms.env=PRODUCT
logging.config=classpath:logback-product.xml
####### 数据库配置 #######
####### Database properties(Configure according to the the environment) #######
spring.datasource.core.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.core.jdbc-url=jdbc:mysql://localhost:3306/powerjob-product?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
spring.datasource.core.username=root
@ -9,11 +9,13 @@ spring.datasource.core.password=No1Bug2Please3!
spring.datasource.core.hikari.maximum-pool-size=20
spring.datasource.core.hikari.minimum-idle=5
####### mongoDB配置，非核心依赖，通过配置 oms.mongodb.enable=false 来关闭 #######
####### MongoDB properties(Non-core configuration properties) #######
####### configure oms.mongodb.enable=false to disable mongodb #######
oms.mongodb.enable=true
spring.data.mongodb.uri=mongodb://localhost:27017/powerjob-product
####### 邮件配置（不需要邮件报警可以删除以下配置来避免报错） #######
####### Email properties(Non-core configuration properties) #######
####### Delete the following code to disable the mail #######
spring.mail.host=smtp.qq.com
spring.mail.username=zqq
spring.mail.password=qqz
@ -21,18 +23,20 @@ spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true
####### 钉钉报警配置（不需要钉钉报警可以删除以下配置来避免报错） #######
####### DingTalk properties(Non-core configuration properties) #######
####### Delete the following code to disable the DingTalk #######
oms.alarm.ding.app-key=
oms.alarm.ding.app-secret=
oms.alarm.ding.agent-id=
####### 资源清理配置 #######
####### Resource cleaning properties #######
oms.instanceinfo.retention=7
oms.container.retention.local=7
oms.container.retention.remote=-1
####### 缓存配置 #######
####### Cache properties #######
oms.instance.metadata.cache.size=2048
####### 精确获取 server 的百分比，0～100，100代表每次 worker 获取 server 都会进行完整的探活流程，不存在脑裂问题，但有性能开销 #######
####### Threshold in precise fetching server(0~100). 100 means full detection of server, in which #######
####### split-brain could be avoided while performance overhead would increase. #######
oms.accurate.select.server.percentage = 50

10
powerjob-server/src/main/resources/application.properties
View File

@ -1,4 +1,4 @@
# http 服务端口
# Http server port
server.port=7700
spring.profiles.active=daily
@ -7,14 +7,14 @@ spring.jpa.open-in-view=false
spring.data.mongodb.repositories.type=none
logging.level.org.mongodb=warn
# 文件上传配置
# Configuration for uploading files.
spring.servlet.multipart.enabled=true
spring.servlet.multipart.file-size-threshold=0
spring.servlet.multipart.max-file-size=209715200
spring.servlet.multipart.max-request-size=209715200
###### PowerJob 自身配置（该配置只允许存在于 application.properties 文件中） ######
# akka ActorSystem 服务端口
###### PowerJob self-owned configuration (The following properties should exist in application.properties only). ######
# Akka ActorSystem port.
oms.akka.port=10086
# 表前缀（默认无表前缀，有需求直接填入表前缀即可，比如 pj_ ）
# Prefix for all tables. Default empty string. Config if you have needs, i.e. pj_
oms.table-prefix=

2
powerjob-server/src/main/resources/banner.txt
View File

@ -10,5 +10,5 @@ ${AnsiColor.GREEN}
${AnsiColor.BRIGHT_RED}
* Maintainer: tengjiqi@gmail.com & PowerJob-Team
* OfficialWebsite: http://www.powerjob.tech/
* SourceCode: https://github.com/KFCFans/PowerJob
* SourceCode: https://github.com/PowerJob/PowerJob
* PoweredBy: SpringBoot${spring-boot.formatted-version} & Akka (v2.6.4)

16
powerjob-server/src/main/resources/logback-dev.xml
View File

@ -1,18 +1,18 @@
<?xml version="1.0"?>
<!-- 本地调试专用，所有日志都从 console 输出-->
<!-- Configuration for local environment, all logs would print in console. -->
<configuration>
<!-- 彩色日志 -->
<!-- 彩色日志依赖的渲染类 -->
<!-- Configure color for logs. -->
<!-- Classes for rendering color. -->
<conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter"/>
<conversionRule conversionWord="wex"
converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter"/>
<conversionRule conversionWord="wEx"
converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter"/>
<!-- 彩色日志格式 -->
<!-- Color log pattern. -->
<property name="CONSOLE_LOG_PATTERN"
value="${CONSOLE_LOG_PATTERN:-%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{20}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>
<!-- Console 输出设置 -->
<!-- Configuration for console output. -->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
@ -20,13 +20,13 @@
</encoder>
</appender>
<!-- 定时调度信息控制台就不输出了，看着就乱 -->
<!-- Quit timing logs, which seems disordered. -->
<logger name="com.github.kfcfans.powerjob.server.service.timing" level="WARN" additivity="false">
<appender-ref ref="CONSOLE" />
<appender-ref ref="CONSOLE"/>
</logger>
<logger name="com.github.kfcfans.powerjob" level="DEBUG" additivity="false">
<appender-ref ref="CONSOLE" />
<appender-ref ref="CONSOLE"/>
</logger>
<root level="INFO">

26
powerjob-server/src/main/resources/logback-product.xml
View File

@ -1,20 +1,20 @@
<?xml version="1.0"?>
<!-- 生产环境日志 -->
<!-- Configuration for production environment. -->
<configuration>
<!--默认配置-->
<!-- Default configuration. -->
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<!--配置控制台(Console)-->
<!-- Configuration for console. -->
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
<!--
日志路径，注意权限问题，否则无法打印日志。
大坑记录：`~/logs`不会在用户目录下创建文件夹，而是在项目目录下创建名为~的文件夹
-->
Log path, pay attention to permission, logs may be unable to generate.
Bug recording: Setting `~/logs`, is unable to create folder in user home directory,
a folder with the name ~ is created in project folder.
-->
<property name="LOG_PATH" value="${user.home}/powerjob-server/logs"/>
<!-- 系统所有异常日志（ERROR）双写 start -->
<!-- Configuration for ERROR logs. All error logs will write twice. -->
<appender name="ERROR_APPENDER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/powerjob-server-error.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
@ -31,9 +31,9 @@
<onMismatch>DENY</onMismatch>
</filter>
</appender>
<!-- 系统所有异常日志（ERROR）双写 end -->
<!-- End of configuration for ERROR logs. -->
<!-- web 访问日志 start -->
<!-- Configuration for Web services. -->
<appender name="WEB_LOG_APPENDER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/powerjob-server-web.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
@ -49,9 +49,9 @@
<logger name="WEB_LOG" level="INFO" additivity="false">
<appender-ref ref="WEB_LOG_APPENDER"/>
</logger>
<!-- web 访问日志 日志 end -->
<!-- End of configuration for Web services. -->
<!-- 系统主日志 start -->
<!-- Configuration for system logs. -->
<appender name="DEFAULT_APPENDER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/powerjob-server-application.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
@ -64,7 +64,7 @@
</encoder>
<append>true</append>
</appender>
<!-- 系统主日志 日志 end -->
<!-- End of configuration for system logs. -->
<root level="INFO">
<appender-ref ref="CONSOLE"/>

2
powerjob-server/src/main/resources/static/js/5.js
View File

File diff suppressed because one or more lines are too long

2
powerjob-server/src/main/resources/static/js/6.js
View File

File diff suppressed because one or more lines are too long

2
powerjob-server/src/main/resources/static/js/9.js
View File

File diff suppressed because one or more lines are too long

2
powerjob-server/src/main/resources/static/js/app.js
View File

File diff suppressed because one or more lines are too long

6
powerjob-worker-agent/pom.xml
View File

@ -10,12 +10,12 @@
<modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-worker-agent</artifactId>
<version>3.4.2</version>
<version>3.4.3</version>
<packaging>jar</packaging>
<properties>
<powerjob.worker.version>3.4.2</powerjob.worker.version>
<powerjob.worker.version>3.4.3</powerjob.worker.version>
<logback.version>1.2.3</logback.version>
<picocli.version>4.3.2</picocli.version>
@ -63,7 +63,7 @@
<executions>
<execution>
<goals>
<goal>repackage</goal><!--可以把依赖的包都打包到生成的Jar包中-->
<goal>repackage</goal>
</goals>
</execution>
</executions>

4
powerjob-worker-agent/src/main/java/com/github/kfcfans/powerjob/worker/MainApplication.java
View File

@ -11,13 +11,13 @@ import picocli.CommandLine.Command;
import picocli.CommandLine.Option;
/**
* 启动类
* powerjob-worker-agent entry
*
* @author tjq
* @since 2020/5/20
*/
@Slf4j
@Command(name = "OhMyAgent", mixinStandardHelpOptions = true, version = "1.2.0", description = "OhMyScheduler-Worker agent")
@Command(name = "OhMyAgent", mixinStandardHelpOptions = true, version = "3.4.3", description = "powerjob-worker agent")
public class MainApplication implements Runnable {
@Option(names = {"-a", "--app"}, description = "worker-agent's name", required = true)

4
powerjob-worker-samples/pom.xml
View File

@ -10,11 +10,11 @@
<modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-worker-samples</artifactId>
<version>3.4.2</version>
<version>3.4.3</version>
<properties>
<springboot.version>2.2.6.RELEASE</springboot.version>
<powerjob.worker.starter.version>3.4.2</powerjob.worker.starter.version>
<powerjob.worker.starter.version>3.4.3</powerjob.worker.starter.version>
<fastjson.version>1.2.68</fastjson.version>
<!-- 部署时跳过该module -->

2
powerjob-worker-samples/src/main/java/com/github/kfcfans/powerjob/samples/tester/StopInstanceTester.java
View File

@ -7,7 +7,7 @@ import org.springframework.stereotype.Component;
/**
* 测试用户反馈的无法停止实例的问题
* https://github.com/KFCFans/PowerJob/issues/37
* https://github.com/PowerJob/PowerJob/issues/37
*
* @author tjq
* @since 2020/7/30

14
powerjob-worker-samples/src/main/resources/application.properties
View File

@ -1,15 +1,13 @@
server.port=8081
spring.jpa.open-in-view=false
########### powerjob-worker 配置（老配置 powerjob.xxx 即将废弃，请使用 powerjob.worker.xxx） ###########
# akka 工作端口，可选，默认 27777
########### PowerJob-worker properties. ###########
# Akka port, default is 27777
powerjob.worker.akka-port=27777
# 接入应用名称，用于分组隔离，推荐填写 本 Java 项目名称
# Application name, used for grouping applications. Recommend to set the same value as project name.
powerjob.worker.app-name=powerjob-agent-test
# 调度服务器地址，IP:Port 或 域名，多值逗号分隔
# Address of PowerJob-server node(s). Ip:port or domain. Multiple addresses should be separated with comma.
powerjob.worker.server-address=127.0.0.1:7700,127.0.0.1:7701
# 持久化方式，可选，默认 disk
# Store strategy of H2 database. disk or memory. Default value is disk.
powerjob.worker.store-strategy=disk
# 返回值最大长度，默认 8096
# Max length of result. Results that are longer than the value will be truncated.
powerjob.worker.max-result-length=4096

4
powerjob-worker-spring-boot-starter/pom.xml
View File

@ -10,11 +10,11 @@
<modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-worker-spring-boot-starter</artifactId>
<version>3.4.2</version>
<version>3.4.3</version>
<packaging>jar</packaging>
<properties>
<powerjob.worker.version>3.4.2</powerjob.worker.version>
<powerjob.worker.version>3.4.3</powerjob.worker.version>
<springboot.version>2.2.6.RELEASE</springboot.version>
</properties>

40
powerjob-worker-spring-boot-starter/src/main/java/com/github/kfcfans/powerjob/worker/autoconfigure/PowerJobAutoConfiguration.java
View File

@ -16,7 +16,7 @@ import java.util.Arrays;
import java.util.List;
/**
* PowerJob 自动装配
* Auto configuration class for PowerJob-worker.
*
* @author songyinyin
* @since 2020/7/26 16:37
@ -32,30 +32,46 @@ public class PowerJobAutoConfiguration {
PowerJobProperties.Worker worker = properties.getWorker();
// 服务器HTTP地址（端口号为 server.port，而不是 ActorSystem port），请勿添加任何前缀（http://）
/*
* Address of PowerJob-server node(s). Do not mistake for ActorSystem port. Do not add
* any prefix, i.e. http://.
*/
CommonUtils.requireNonNull(worker.getServerAddress(), "serverAddress can't be empty!");
List<String> serverAddress = Arrays.asList(worker.getServerAddress().split(","));
// 1. 创建配置文件
/*
* Create OhMyConfig object for setting properties.
*/
OhMyConfig config = new OhMyConfig();
// 端口配置，支持随机端口
/*
* Configuration of worker port. Random port is enabled when port is set with non-positive number.
*/
int port = worker.getAkkaPort();
if (port <= 0) {
port = NetUtils.getRandomPort();
}
config.setPort(port);
// appName，需要提前在控制台注册，否则启动报错
/*
* appName, name of the application. Applications should be registered in advance to prevent
* error. This property should be the same with what you entered for appName when getting
* registered.
*/
config.setAppName(worker.getAppName());
config.setServerAddress(serverAddress);
// 如果没有大型 Map/MapReduce 的需求，建议使用内存来加速计算
// 有大型 Map/MapReduce 需求，可能产生大量子任务（Task）的场景，请使用 DISK，否则妥妥的 OutOfMemory
/*
* For non-Map/MapReduce tasks, {@code memory} is recommended for speeding up calculation.
* Map/MapReduce tasks may produce batches of subtasks, which could lead to OutOfMemory
* exception or error, {@code disk} should be applied.
*/
config.setStoreStrategy(worker.getStoreStrategy());
// 启动测试模式，true情况下，不再尝试连接 server 并验证appName
/*
* When enabledTestMode is set as true, PowerJob-worker no longer connects to PowerJob-server
* or validate appName.
*/
config.setEnableTestMode(worker.isEnableTestMode());
// 2. 创建 Worker 对象，设置配置文件
/*
* Create OhMyWorker object and set properties.
*/
OhMyWorker ohMyWorker = new OhMyWorker();
ohMyWorker.setConfig(config);
return ohMyWorker;

32
powerjob-worker-spring-boot-starter/src/main/java/com/github/kfcfans/powerjob/worker/autoconfigure/PowerJobProperties.java
View File

@ -9,7 +9,7 @@ import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.DeprecatedConfigurationProperty;
/**
* PowerJob 配置项
* PowerJob properties configuration class.
*
* @author songyinyin
* @since 2020/7/26 16:37
@ -90,37 +90,47 @@ public class PowerJobProperties {
}
/**
* 客户端 配置项
* Powerjob worker configuration properties.
*/
@Setter
@Getter
public static class Worker {
/**
* 应用名称，需要提前在控制台注册，否则启动报错
* Name of application, String type. Total length of this property should be no more than 255
* characters. This is one of the required properties when registering a new application. This
* property should be assigned with the same value as what you entered for the appName.
*/
private String appName;
/**
* 启动 akka 端口
* Akka port of Powerjob-worker, optional value. Default value of this property is 27777.
* If multiple PowerJob-worker nodes were deployed, different, unique ports should be assigned.
*/
private int akkaPort = RemoteConstant.DEFAULT_WORKER_PORT;
/**
* 调度服务器地址，ip:port 或 域名，多个用英文逗号分隔
* Address(es) of Powerjob-server node(s). Ip:port or domain.
* Example of single Powerjob-server node:
* <p>
* 127.0.0.1:7700
* </p>
* Example of Powerjob-server cluster:
* <p>
* 192.168.0.10:7700,192.168.0.11:7700,192.168.0.12:7700
* </p>
*/
private String serverAddress;
/**
* 本地持久化方式，默认使用磁盘
* Local store strategy for H2 database. {@code disk} or {@code memory}.
*/
private StoreStrategy storeStrategy = StoreStrategy.DISK;
/**
* 最大返回值长度，超过会被截断
* {@link ProcessResult}#msg 的最大长度
* Max length of response result. Result that is longer than the value will be truncated.
* {@link ProcessResult} max length for #msg
*/
private int maxResultLength = 8096;
/**
* 启动测试模式，true情况下，不再尝试连接 server 并验证appName。
* true -> 用于本地写单元测试调试； false -> 默认值，标准模式
* If test mode is set as true, Powerjob-worker no longer connects to the server or validates appName.
* Test mode is used for conditions that your have no powerjob-server in your develop env so you can't startup the application
*/
private boolean enableTestMode = false;
}

12
powerjob-worker-spring-boot-starter/src/main/resources/META-INF/spring-configuration-metadata.json
View File

@ -16,39 +16,39 @@
{
"name": "powerjob.worker.akka-port",
"type": "java.lang.Integer",
"description": "启动 akka 端口",
"description": "Akka port of PowerJob-worker",
"sourceType": "com.github.kfcfans.powerjob.worker.autoconfigure.PowerJobProperties$Worker"
},
{
"name": "powerjob.worker.app-name",
"type": "java.lang.String",
"description": "应用名称，需要提前在控制台注册，否则启动报错",
"description": "Name of application. Register in PowerJob-console to prevent error.",
"sourceType": "com.github.kfcfans.powerjob.worker.autoconfigure.PowerJobProperties$Worker"
},
{
"name": "powerjob.worker.enable-test-mode",
"type": "java.lang.Boolean",
"description": "启动测试模式，true情况下，不再尝试连接 server 并验证appName。 true -> 用于本地写单元测试调试； false -> 默认值，标准模式",
"description": "Whether to enable test mode. In test mode, worker will not connect to server.",
"sourceType": "com.github.kfcfans.powerjob.worker.autoconfigure.PowerJobProperties$Worker",
"defaultValue": false
},
{
"name": "powerjob.worker.max-result-length",
"type": "java.lang.Integer",
"description": "最大返回值长度，超过会被截断 {@link ProcessResult}#msg 的最大长度",
"description": "Max length for {@link ProcessResult}#msg, result longer than this property will be truncated.",
"sourceType": "com.github.kfcfans.powerjob.worker.autoconfigure.PowerJobProperties$Worker",
"defaultValue": 8096
},
{
"name": "powerjob.worker.server-address",
"type": "java.lang.String",
"description": "调度服务器地址，ip:port 或 域名，多个用英文逗号分隔",
"description": "PowerJob-server node(s) address. Ip:port or domain, multiple addresses should be separated with comma",
"sourceType": "com.github.kfcfans.powerjob.worker.autoconfigure.PowerJobProperties$Worker"
},
{
"name": "powerjob.worker.store-strategy",
"type": "com.github.kfcfans.powerjob.worker.common.constants.StoreStrategy",
"description": "本地持久化方式，默认使用磁盘",
"description": "Local store strategy, disk or memory",
"sourceType": "com.github.kfcfans.powerjob.worker.autoconfigure.PowerJobProperties$Worker"
}
],

4
powerjob-worker-spring-boot-starter/src/test/java/com/github/kfcfans/powerjob/worker/autoconfigure/PowerJobAutoConfigurationTest.java
View File

@ -1,7 +1,7 @@
package com.github.kfcfans.powerjob.worker.autoconfigure;
import com.github.kfcfans.powerjob.worker.OhMyWorker;
import org.junit.Assert;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
@ -16,7 +16,7 @@ class PowerJobAutoConfigurationTest {
void testAutoConfiguration() {
ConfigurableApplicationContext run = SpringApplication.run(PowerJobAutoConfigurationTest.class);
OhMyWorker worker = run.getBean(OhMyWorker.class);
Assert.assertNotNull(worker);
Assertions.assertNotNull(worker);
}
}

1
powerjob-worker-spring-boot-starter/src/test/resources/application.properties
View File

@ -1,2 +1 @@
powerjob.enable-test-mode=true

4
powerjob-worker/pom.xml
View File

@ -10,12 +10,12 @@
<modelVersion>4.0.0</modelVersion>
<artifactId>powerjob-worker</artifactId>
<version>3.4.2</version>
<version>3.4.3</version>
<packaging>jar</packaging>
<properties>
<spring.version>5.2.4.RELEASE</spring.version>
<powerjob.common.version>3.4.2</powerjob.common.version>
<powerjob.common.version>3.4.3</powerjob.common.version>
<h2.db.version>1.4.200</h2.db.version>
<hikaricp.version>3.4.2</hikaricp.version>
<junit.version>5.6.1</junit.version>

32
powerjob-worker/src/main/java/com/github/kfcfans/powerjob/worker/common/OhMyConfig.java
View File

@ -10,7 +10,7 @@ import lombok.Setter;
import java.util.List;
/**
* Worker 配置文件
* The powerjob-worker's configuration
*
* @author tjq
* @since 2020/3/16
@ -19,34 +19,38 @@ import java.util.List;
@Setter
public class OhMyConfig {
/**
* 应用名称
* AppName, recommend to use the name of this project
* Applications should be registered by powerjob-console in advance to prevent error.
*/
private String appName;
/**
* 启动端口
* Worker port
* Random port is enabled when port is set with non-positive number.
*/
private int port = RemoteConstant.DEFAULT_WORKER_PORT;
/**
* 调度服务器地址，ip:port 或 域名
* Address of powerjob-server node(s)
* Do not mistake for ActorSystem port. Do not add any prefix, i.e. http://.
*/
private List<String> serverAddress = Lists.newArrayList();
/**
* 本地持久化方式，默认使用磁盘
*/
private StoreStrategy storeStrategy = StoreStrategy.DISK;
/**
* 最大返回值长度，超过会被截断
* {@link ProcessResult}#msg 的最大长度
* Max length of response result. Result that is longer than the value will be truncated.
* {@link ProcessResult} max length for #msg
*/
private int maxResultLength = 8096;
/**
* 用户自定义上下文对象，该值会被透传到 TaskContext#userContext 属性
* 使用场景：容器脚本Java处理器需要使用oms-worker宿主应用的Spring Bean，可在此处传入 ApplicationContext，在Processor中获取 bean
* User-defined context object, which is passed through to the TaskContext#userContext property
* Usage Scenarios: The container Java processor needs to use the Spring bean of the host application, where you can pass in the ApplicationContext and get the bean in the Processor
*/
private Object userContext;
/**
* 启动测试模式，true情况下，不再尝试连接 server 并验证appName
* true -> 用于本地写单元测试调试； false -> 默认值，标准模式
* Internal persistence method, DISK or MEMORY
* Normally you don't need to care about this configuration
*/
private StoreStrategy storeStrategy = StoreStrategy.DISK;
/**
* If test mode is set as true, Powerjob-worker no longer connects to the server or validates appName.
* Test mode is used for conditions that your have no powerjob-server in your develop env so you can't startup the application
*/
private boolean enableTestMode = false;
}

2
powerjob-worker/src/main/java/com/github/kfcfans/powerjob/worker/common/PowerBannerPrinter.java
View File

@ -24,7 +24,7 @@ public final class PowerBannerPrinter {
"\n" +
"* Maintainer: tengjiqi@gmail.com & PowerJob-Team\n" +
"* OfficialWebsite: http://www.powerjob.tech/\n" +
"* SourceCode: https://github.com/KFCFans/PowerJob\n" +
"* SourceCode: https://github.com/PowerJob/PowerJob\n" +
"\n";
public static void print() {

4
powerjob-worker/src/main/java/com/github/kfcfans/powerjob/worker/common/constants/StoreStrategy.java
View File

@ -1,6 +1,7 @@
package com.github.kfcfans.powerjob.worker.common.constants;
import lombok.AllArgsConstructor;
import lombok.Getter;
/**
* 持久化策略
@ -8,11 +9,12 @@ import lombok.AllArgsConstructor;
* @author tjq
* @since 2020/4/14
*/
@Getter
@AllArgsConstructor
public enum StoreStrategy {
DISK("磁盘"),
MEMORY("内存");
private String des;
private final String des;
}

6
powerjob-worker/src/main/java/com/github/kfcfans/powerjob/worker/core/tracker/task/FrequentTaskTracker.java
View File

@ -88,7 +88,7 @@ public class FrequentTaskTracker extends TaskTracker {
// 2. 启动任务发射器
launcher = new Launcher();
if (timeExpressionType == TimeExpressionType.FIX_RATE) {
if (timeExpressionType == TimeExpressionType.FIXED_RATE) {
// 固定频率需要设置最小间隔
if (timeParams < MIN_INTERVAL) {
throw new PowerJobException("time interval too small, please set the timeExpressionInfo >= 1000");
@ -172,7 +172,7 @@ public class FrequentTaskTracker extends TaskTracker {
// 判断是否超出最大执行实例数
if (maxInstanceNum > 0) {
if (timeExpressionType == TimeExpressionType.FIX_RATE) {
if (timeExpressionType == TimeExpressionType.FIXED_RATE) {
if (subInstanceId2TimeHolder.size() > maxInstanceNum) {
log.warn("[FQTaskTracker-{}] cancel to launch the subInstance({}) due to too much subInstance is running.", instanceId, subInstanceId);
processFinishedSubInstance(subInstanceId, false, "TOO_MUCH_INSTANCE");
@ -368,7 +368,7 @@ public class FrequentTaskTracker extends TaskTracker {
taskPersistenceService.deleteAllSubInstanceTasks(instanceId, subInstanceId);
// FIX_DELAY 则调度下次任务
if (timeExpressionType == TimeExpressionType.FIX_DELAY) {
if (timeExpressionType == TimeExpressionType.FIXED_DELAY) {
scheduledPool.schedule(launcher, timeParams, TimeUnit.MILLISECONDS);
}
}

4
powerjob-worker/src/main/java/com/github/kfcfans/powerjob/worker/core/tracker/task/TaskTracker.java
View File

@ -115,8 +115,8 @@ public abstract class TaskTracker {
try {
TimeExpressionType timeExpressionType = TimeExpressionType.valueOf(req.getTimeExpressionType());
switch (timeExpressionType) {
case FIX_RATE:
case FIX_DELAY:return new FrequentTaskTracker(req);
case FIXED_RATE:
case FIXED_DELAY:return new FrequentTaskTracker(req);
default:return new CommonTaskTracker(req);
}
} catch (Exception e) {

4
powerjob-worker/src/test/java/com/github/kfcfans/powerjob/FrequentTaskTrackerTest.java
View File

@ -41,13 +41,13 @@ public class FrequentTaskTrackerTest {
@Test
public void testFixRateJob() throws Exception {
remoteTaskTracker.tell(TestUtils.genServerScheduleJobReq(ExecuteType.STANDALONE, TimeExpressionType.FIX_RATE), null);
remoteTaskTracker.tell(TestUtils.genServerScheduleJobReq(ExecuteType.STANDALONE, TimeExpressionType.FIXED_RATE), null);
Thread.sleep(5000000);
}
@Test
public void testFixDelayJob() throws Exception {
remoteTaskTracker.tell(TestUtils.genServerScheduleJobReq(ExecuteType.MAP_REDUCE, TimeExpressionType.FIX_DELAY), null);
remoteTaskTracker.tell(TestUtils.genServerScheduleJobReq(ExecuteType.MAP_REDUCE, TimeExpressionType.FIXED_DELAY), null);
Thread.sleep(5000000);
}
}

4
powerjob-worker/src/test/java/com/github/kfcfans/powerjob/TestUtils.java
View File

@ -32,8 +32,8 @@ public class TestUtils {
req.setTimeExpressionType(timeExpressionType.name());
switch (timeExpressionType) {
case CRON:req.setTimeExpression("0 * * * * ? ");
case FIX_RATE:
case FIX_DELAY:req.setTimeExpression("5000");
case FIXED_RATE:
case FIXED_DELAY:req.setTimeExpression("5000");
}
switch (executeType) {