Amazon EKS によるスケーラブルな CTR 予測システムを導入した話
job ~ worker system
userID + adID ~ boolean
CPC(Cost Per Click)
lightGBMSpark on k8s
1h window
10~3 億レコード EC2 => 10 hours
goal: 1h + scaling
framework version
cl / cd
learn / estimate using different resource
container
helm
Node Group Autoscaler
spark => s3 => sqs => pod scaler + cluster auto scaler => message => download model => estimate => s3 => delete message => scale down
k8s job X
- kick outside cluster like lambda
- parallelism not flexible
pub/sub + Fan-Out OK
onErr => wait until message visibility
available message >1 scale up available message =0 scale down
python signal SIGTERM
config injection helm => value.yml + stg|pro|dev.yml
save in/out container in:
- sync with git hash
- rolling update
- need build + deploy out:
- no need to build + deploy
- simple
based on latest <- only see the results
Azure Kubernetes Service で実現する超低予算＆(ほぼ)フルマネージド＆本格的な WordPress 環境
- azure.sios.jp
- tech-lab.sios.jp
admin 2 + frontend 2 => 4 clusters
Wp supter Cache
Azure Load Balancer
nginx ingress
Azure Container Registry
ptrStop + SIGTERM => 3s+ 30s+ apachectl graceful-stop => termination
SMB or NFS or VM NPS
Blackfire <- profiler
35000/月
app gateway <- was in preview
azure backup to backup vm