-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] [OSS Checkpoint] OSS checkpoint not working #6779
Labels
Comments
I also encountered this issue, but now I've been utilizing Ali OSS HDFS to store my checkpoints successfully. I suggest you consider testing this solution! |
我存储成功了,但是如果重启docker的话不会读取checkpoint,所有任务都丢失了 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Search before asking
What happened
When I use Aliyun OSS to store the checkpoint, the configuration is as follows, the storage is successful, and I can find the checkpoint from the OSS file. I deployed the seatunnel server using Docker. When I restart, the real-time synchronization tasks I previously established disappear, and the server does not reload historical tasks from the checkpoint. Does seatunel have the ability to reload historical tasks from checkpoint?
seatunnel:
engine:
history-job-expire-minutes: 1440
backup-count: 1
queue-type: blockingqueue
print-execution-info-interval: 60
print-job-metrics-info-interval: 60
slot-service:
dynamic-slot: true
checkpoint:
interval: 10000
timeout: 600000
storage:
type: hdfs
max-retained: 3
plugin-config:
storage.type: oss
namespace: /tmp/seatunnel/checkpoint_snapshot
oss.bucket: oss://xxx
fs.oss.accessKeyId: xxx
fs.oss.accessKeySecret: xxx
fs.oss.endpoint: oss-cn-hangzhou.aliyuncs.com
SeaTunnel Version
2.3.3
SeaTunnel Config
Running Command
Error Exception
Zeta or Flink or Spark Version
zeta 2.3.3
Java or Scala Version
1.8
Screenshots
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: