rssLink RSS for all categories
 
icon_red
icon_green
icon_red
icon_red
icon_blue
icon_green
icon_green
icon_red
icon_red
icon_red
icon_orange
icon_green
icon_green
icon_green
icon_green
icon_blue
icon_red
icon_orange
icon_red
icon_red
icon_red
icon_red
icon_green
icon_red
icon_red
icon_red
icon_red
icon_orange
icon_green
 

FS#14172 — FS#20636 — VPS CLOUD 2016 - GRA CEPH

Attached to Project— Cloud
Incident
Cloud
In progress
0%
There has been an issue on CEPH cluster in GRA for VPS cloud. The Ceph team is working on it.
Comment by OVH - Monday, 03 October 2016, 17:47PM

Due to a problem with the storage node, we experience degraded performance


Comment by OVH - Monday, 03 October 2016, 17:49PM

Impacted VPS:

vps197911
vps206399
vps206586
vps206564
vps206842
vps206869
vps207564
vps207605
vps211998
vps211638
vps223221
vps223459
vps223681
vps221883
vps224028
vps224059
vps226786
vps226780
vps228207
vps229438
vps232361
vps236018
vps237680
vps238780
vps260726
vps260855
vps261813
vps265864
vps279766
vps281192
vps282686
vps283076
vps283736
vps284212
vps284286
vps284544
vps284635
vps283679
vps279192
vps211699
vps212060
vps212450
vps213658
vps222961
vps223989
vps224055
vps224103
vps224416
vps223144
vps225963
vps227037
vps227413
vps227673
vps227412
vps232889
vps233892
vps239022
vps239443
vps246546
vps257528
vps263390
vps266128
vps271434
vps273461
vps274934
vps234661
vps226740
vps282231
vps283437
vps283818
vps284427
vps282728
vps298090
vps199520
vps202585
vps208281
vps211446
vps212086
vps214714
vps216452
vps216717
vps216677
vps210366
vps222251
vps222932
vps223026
vps223079
vps223128
vps217164
vps226650
vps227846
vps230531
vps231307
vps232136
vps225449
vps234431
vps235110
vps238284
vps238958
vps239059
vps239195
vps241334
vps249284
vps256735
vps260171
vps261059
vps263367
vps264360
vps272917
vps279386
vps281028
vps282079
vps282129
vps282271
vps284040
vps285059
vps285267
vps285611
vps279396
vps285937
vps286396
vps287006
vps287041
vps289848
vps298011
vps287014
vps258201
vps250130
vps195469
vps204386
vps205751
vps206217
vps209843
vps219802
vps219957
vps211305
vps234611
vps262540
vps268983
vps279744
vps280602
vps195217
vps285529
vps285935
vps288129
vps291126
vps299007
vps300211
vps214403
vps224096
vps231732
vps235309
vps244379
vps250000
vps235719
vps263945
vps283633
vps283880
vps283923
vps283702
vps283708
vps285537
vps286064
vps211953
vps298053
vps299900
vps283300
vps205905
vps206074
vps206352
vps207521
vps209088
vps214327
vps215300
vps215746
vps214992
vps215650
vps218141
vps220462
vps220929
vps220927
vps220931
vps220409
vps221934
vps224587
vps227565
vps228661
vps229016
vps229292
vps229340
vps229420
vps230812
vps233793
vps233746
vps235271
vps235796
vps236084
vps236208
vps236797
vps240224
vps238932
vps228495
vps245749
vps249011
vps251897
vps260563
vps260711
vps261915
vps263244
vps278783
vps279194
vps281368
vps284207
vps284936
vps285022
vps301348
vps206783
vps211965
vps216462
vps216508
vps216852
vps227054
vps227242
vps227886
vps229964
vps228441
vps233297
vps243682
vps243662
vps255451
vps277995
vps279040
vps282935
vps282959
vps283025
vps286360
vps211614
vps225170
vps225422
vps225445
vps231462
vps231831
vps233121
vps233457
vps244815
vps252831
vps255180
vps217038
vps285310
vps287473
vps263575
vps297786
vps298061
vps193373
vps194187
vps196944
vps197509
vps196943
vps206515
vps206727
vps206465
vps211591
vps211705
vps211921
vps211941
vps214049
vps214154
vps213458
vps216475
vps216997
vps224655
vps232024
vps236484
vps239774
vps239899
vps245296
vps247637
vps260144
vps261062
vps260590
vps268044
vps276306
vps278865
vps287772
vps203087
vps294032
vps297511
vps297800
vps206177
vps207902
vps214408
vps195994
vps215011
vps225360
vps260597
vps260603
vps260601
vps269099
vps275411
vps230360
vps216481
vps294979
vps295030
vps203699
vps211701
vps230740
vps204647
vps214867
vps224995
vps235057
vps226773
vps236729
vps238381
vps208092
vps257608
vps263000
vps266970
vps268113
vps262085
vps283123
vps286754
vps288873
vps259472
vps301389
vps204278
vps204172
vps207688
vps214221
vps215906
vps216399
vps213064
vps217103
vps216305
vps217451
vps220712
vps224250
vps224685
vps224705
vps226939
vps232635
vps233514
vps233002
vps237689
vps240616
vps235578
vps253533
vps263254
vps266179
vps273091
vps274217
vps275029
vps278180
vps278869
vps279002
vps281372
vps281466
vps281523
vps282187
vps282281
vps282579
vps286487
vps286791
vps281930
vps285930
vps282236
vps287553
vps295492
vps295587
vps231397
vps234555
vps224613
vps301861
vps302657
vps272198
vps262800
vps202577
vps203266
vps208918
vps210037
vps211239
vps214744
vps216839
vps228255
vps231183
vps236407
vps201374
vps234080
vps260187
vps280940
vps279646
vps250156
vps205851
vps214010
vps216347
vps219023
vps220662
vps221011
vps222205
vps223982
vps223983
vps218939
vps221860
vps225328
vps229335
vps229503
vps239108
vps243289
vps244576
vps248386
vps202712
vps213031
vps213962
vps219450
vps219306
vps222115
vps230003
vps219024
vps235265
vps237862
vps240778
vps254915
vps288191
vps286954
vps295947
vps296166
vps300486
vps258824
vps212673
vps216024
vps225998
vps233228
vps239306
vps240090
vps247622
vps253399
vps282365
vps286742
vps286906
vps287494
vps295817
vps282486
vps299888
vps302474
vps258827
vps201534
vps202149
vps209705
vps222754
vps226121
vps203135
vps231524
vps233912
vps255604
vps264512
vps265701
vps271114
vps279938
vps283360
vps283927
vps285470
vps285582
vps287711
vps266956
vps205281
vps210462
vps210479
vps210854
vps213776
vps204615
vps215763
vps216177
vps223345
vps205674
vps226978
vps227253
vps227409
vps228563
vps231986
vps237611
vps237771
vps243330
vps260951
vps263308
vps266547
vps271928
vps278064
vps278823
vps279367
vps262829
vps280961
vps285747
vps286780
vps286790
vps296636
vps203359
vps220620
vps222195
vps224964
vps225094
vps236542
vps236741
vps236749
vps239840
vps238433
vps243770
vps289640
vps255086
vps200671
vps202733
vps210189
vps210461
vps210605
vps211717
vps210188
vps226469
vps238899
vps240405
vps250455
vps278171
vps209548
vps284117
vps282999
vps290471
vps203582
vps297646
vps203254
vps204601
vps216698
vps223162
vps223203
vps224785
vps222353
vps226611
vps232999
vps234564
vps238225
vps270176
vps281889
vps282175
vps282350
vps223636
vps295809
vps296398
vps212731
vps212943
vps213059
vps213331
vps215238
vps216723
vps228846
vps229197
vps229226
vps230362
vps230717
vps196301
vps233642
vps240722
vps243343
vps236768
vps245140
vps203103
vps238058
vps263060
vps281209
vps242390
vps288268
vps297506
vps278042
vps301417
vps301244
vps301729
vps301833
vps302050
vps302080
vps302127
vps302401
vps302466
vps255014
vps288347
vps302674
vps281269
vps197908
vps211011
vps217631
vps217754
vps217891
vps218109
vps217444
vps219385
vps219594
vps221956
vps13938
vps229911
vps230292
vps234680
vps235650
vps218179
vps237553
vps237737
vps239046
vps239280
vps233115
vps242967
vps261262
vps262038
vps237602
vps277539
vps282619
vps283648
vps283779
vps284064
vps282682
vps284422
vps284798
vps284885
vps240566
vps298009
vps299241
vps299883
vps199542
vps202680
vps204076
vps204284
vps204423
vps204490
vps204515
vps213048
vps215026
vps213178
vps209491
vps217392
vps222563
vps223151
vps223304
vps223406
vps223524
vps223654
vps224413
vps224642
vps210333
vps226821
vps227318
vps237427
vps242452
vps252644
vps217718
vps235731
vps259111
vps261048
vps270312
vps271875
vps220038
vps281588
vps223589
vps283008
vps282819
vps291297
vps204027
vps207823
vps209098
vps209930
vps214458
vps200110
vps217115
vps219162
vps223148
vps228210
vps228459
vps229125
vps233849
vps235840
vps250353
vps260629
vps260705
vps269347
vps260824
vps220835
vps260016
vps278575
vps283599
vps284303
vps293701
vps294018
vps295825
vps238985
vps301675
vps302609
vps203916
vps207239
vps210729
vps211463
vps217272
vps216897
vps218208
vps220130
vps224604
vps224896
vps216686
vps227958
vps228684
vps229372
vps229961
vps238258
vps238994
vps240654
vps251066
vps210746
vps260523
vps279794
vps279717
vps281292
vps287688
vps235023
vps291073
vps291669
vps233555
vps199683
vps208388
vps208231
vps209617
vps209887
vps216466
vps220907
vps222190
vps223734
vps224115
vps226391
vps227051
vps228832
vps232141
vps233314
vps178109
vps237100
vps237694
vps239105
vps236445
vps241337
vps241646
vps247977
vps250325
vps222990
vps250786
vps250272
vps255684
vps261423
vps262426
vps262695
vps263317
vps269216
vps271306
vps273950
vps274931
vps279366
vps280895
vps281774
vps281827
vps283002
vps242633
vps283174
vps283285
vps230869
vps272515
vps287088
vps287784
vps287841
vps287844
vps287922
vps301545
vps199679
vps200653
vps201378
vps205421
vps205699
vps208187
vps208331
vps208356
vps208442
vps208429
vps208513
vps210239
vps210304
vps210852
vps210999
vps211326
vps211346
vps213089
vps218065
vps222181
vps228022
vps228836
vps229018
vps229190
vps230703
vps231558
vps234320
vps234508
vps212519
vps244865
vps211455
vps268686
vps268687
vps278053
vps224841
vps288270
vps290016
vps297363
vps297373
vps202163
vps210574
vps223303
vps223940
vps228175
vps228567
vps232048
vps236416
vps236817
vps231364
vps207923
vps261152
vps206740
vps276015
vps283299
vps284061
vps293715
vps205325
vps218751
vps224346
vps226412
vps227097
vps227260
vps231436
vps236207
vps236522
vps236494
vps239783
vps249344
vps255864
vps260706
vps263323
vps263564
vps281291
vps283824
vps283937
vps283959
vps286329
vps286556
vps288340
vps288470
vps291877
vps274211
vps292231
vps292757
vps298003
vps301700
vps202426
vps202652
vps208193
vps208229
vps208510
vps208855
vps209016
vps209869
vps221010
vps221916
vps224338
vps226646
vps227432
vps228319
vps228685
vps229288
vps231576
vps234523
vps234536
vps206514
vps235112
vps237708
vps238172
vps238744
vps238848
vps238793
vps238900
vps238761
vps239155
vps239307
vps239369
vps239870
vps240157
vps113765
vps239088
vps219276
vps245952
vps253213
vps258698
vps262260
vps261095
vps266453
vps269156
vps269761
vps275092
vps209822
vps210406
vps278026
vps278918
vps279747
vps279687
vps281900
vps282878
vps282170
vps282991
vps283084
vps283040
vps283099
vps283056
vps285603
vps283313
vps287167
vps287193
vps287552
vps287658
vps287679
vps293025
vps293643
vps287523
vps286185
vps224603
vps298435
vps284094
vps202045
vps192916
vps206403
vps230149
vps230176
vps230612
vps230480
vps236100
vps240034
vps240476
vps242138
vps247862
vps217788
vps269710
vps279148
vps292101
vps194189
vps207338
vps217812
vps218716
vps223695
vps224357
vps230250
vps231376
vps236537
vps240078
vps281491
vps281655
vps278695
vps218624
vps287307
vps282078
vps292086
vps301632
vps204260
vps210955
vps223629
vps224974
vps229629
vps229928
vps229029
vps232006
vps233028
vps234538
vps236573
vps221359
vps249004
vps251000
vps251124
vps262479
vps262551
vps266602
vps278404
vps279131
vps281034
vps286346
vps286494
vps288525
vps299355
vps302224
vps302250
vps129476
vps302507
vps260917
vps203624
vps204099
vps204815
vps205287
vps205441
vps206355
vps206043
vps210173
vps210237
vps210310
vps213607
vps214250
vps214487
vps214134
vps214622
vps214661
vps214751
vps214776
vps215659
vps216227
vps216345
vps216377
vps216588
vps213553
vps216410
vps219056
vps219180
vps217494
vps220356
vps225509
vps224178
vps225508
vps235170
vps239067
vps240173
vps202498
vps214796
vps255588
vps238420
vps260045
vps260399
vps260484
vps260876
vps262591
vps263589
vps253478
vps266025
vps231638
vps214685
vps208730
vps226237
vps220215
vps280271
vps280478
vps281371
vps283678
vps288732
vps288950
vps293441
vps295977
vps256562
vps300295
vps297271
vps222744
vps227040
vps227258
vps227337
vps227584
vps226862
vps223327
vps234131
vps237668
vps239394
vps253185
vps253496
vps271577
vps273116
vps278285
vps278445
vps278447
vps279391
vps280156
vps280652
vps289654
vps281467
vps300951
vps207273
vps211093
vps213019
vps213491
vps215050
vps220345
vps220425
vps219708
vps222281
vps222842
vps224289
vps226449
vps227292
vps227223
vps220113
vps232104
vps233792
vps234395
vps232298
vps219807
vps250437
vps256493
vps260533
vps278410
vps259534
vps282726
vps287108
vps296372
vps301115
vps195866
vps203190
vps224000
vps224969
vps228519
vps230378
vps234626
vps238485
vps240382
vps262430
vps262691
vps262782
vps263318
vps264409
vps264408
vps269072
vps285552
vps301332
vps200156
vps200656
vps202134
vps203766
vps204679
vps204831
vps205852
vps204571
vps217109
vps225580
vps229404
vps230778
vps227222
vps158013
vps236478
vps239940
vps240085
vps261889
vps272416
vps277782
vps278429
vps278570
vps277900
vps278829
vps281031
vps278509
vps280739
vps285902
vps290311
vps203947
vps282915
vps279781
vps206500
vps206732
vps203331
vps214214
vps216064
vps216538
vps221131
vps223696
vps225953
vps230036
vps230120
vps232749
vps235330
vps238421
vps239004
vps240821
vps249706
vps250555
vps255545
vps227254
vps198525
vps258331
vps205460
vps275443
vps278383
vps282961
vps282994
vps283029
vps283610
vps285110
vps287649
vps298243
vps201558
vps202400
vps204100
vps204344
vps205772
vps206194
vps210272
vps210687
vps210697
vps211014
vps215108
vps206067
vps217008
vps217077
vps218050
vps223970
vps224560
vps224965
vps225335
vps225384
vps225410
vps227129
vps228564
vps228565
vps230224
vps231572
vps226999
vps232060
vps232374
vps232649
vps235230
vps237728
vps236490
vps236558
vps242587
vps247734
vps251312
vps258424
vps258690
vps262559
vps262908
vps260718
vps266252
vps233449
vps266988
vps268049
vps260145
vps269869
vps271656
vps271340
vps274483
vps278066
vps280133
vps280456
vps283317
vps283603
vps54197
vps285640
vps286065
vps162804
vps286146
vps289949
vps291879
vps294819
vps296261
vps296905
vps295154
vps278405
vps97039
vps301055
vps302393
vps237595
vps284001
vps226686
vps204082
vps205296
vps208217
vps208483
vps214325
vps217085
vps207594
vps218152
vps218801
vps220337
vps220338
vps220358
vps222235
vps226676
vps227234
vps227937
vps228990
vps233552
vps234190
vps236479
vps236790
vps248531
vps255587
vps259894
vps261186
vps231273
vps262227
vps262625
vps261005
vps271298
vps279955
vps281040
vps281451
vps279076
vps284626
vps284629
vps284922
vps287106
vps296449
vps297125
vps286143
vps302665
vps302666
vps287609
vps201706
vps208814
vps208616
vps208958
vps209897
vps218289
vps222816
vps221476
vps225979
vps226452
vps228232
vps228638
vps232039
vps233415
vps234653
vps234656
vps234658
vps238096
vps237050
vps240376
vps250332
vps253722
vps255392
vps260291
vps260387
vps276641
vps281958
vps281318
vps283251
vps287016
vps278468
vps226482
vps198951
vps201101
vps207442
vps207852
vps207875
vps209055
vps209618
vps211835
vps212912
vps216346
vps217069
vps216451
vps216511
vps219997
vps221508
vps222460
vps223110
vps223257
vps223753
vps223195
vps226083
vps230421
vps231361
vps231384
vps231561
vps231722
vps232022
vps234837
vps237674
vps237675
vps238113
vps238696
vps240008
vps243862
vps232368
vps246285
vps237428
vps248038
vps252223
vps251686
vps223253
vps250530
vps255838
vps258677
vps240752
vps260774
vps263319
vps265503
vps239814
vps279240
vps281387
vps281702
vps283868
vps284429
vps284412
vps282745
vps287865
vps296511
vps297312
vps300913
vps301620
vps301756
vps213076
vps214872
vps214638
vps215352
vps214831
vps216166
vps217266
vps217030
vps227966
vps234151
vps236769
vps234169
vps258815
vps270932
vps271459
vps277217
vps280377
vps284924
vps219834
vps230210
vps239960
vps255634
vps258490
vps194886
vps209631
vps258529
vps258995
vps281763
vps291271
vps296254
vps298004
vps301080
vps301908
vps302064
vps302220
vps302644
vps216760
vps224404
vps224469
vps225960
vps226987
vps234576
vps236087
vps236350
vps237234
vps234192
vps250397
vps267981
vps278390
vps281429
vps286103
vps293769
vps200979
vps204890
vps205212
vps208548
vps221932
vps222066
vps222265
vps224656
vps225030
vps225287
vps218902
vps211337
vps230895
vps232012
vps232194
vps232460
vps240174
vps253739
vps256693
vps258184
vps260609
vps260630
vps261160
vps260360
vps278126
vps280616
vps280649
vps283561
vps285224
vps285358
vps282242
vps285390
vps287317
vps290609
vps285260
vps217385
vps300161
vps285628
vps212762
vps213370
vps213729
vps215012
vps215536
vps215709
vps215815
vps215123
vps225151
vps227435
vps233227
vps234947
vps245544
vps246289
vps249723
vps282553
vps287347
vps291878
vps205206
vps205285
vps210801
vps213703
vps217736
vps219740
vps220519
vps221256
vps224162
vps224347
vps225005
vps223747
vps225382
vps224958
vps223179
vps232546
vps233907
vps235799
vps239883
vps238863
vps234233
vps250565
vps247583
vps268756
vps270269
vps281641
vps283237
vps289153
vps220703
vps296396
vps298010
vps213014
vps213051
vps214119
vps214995
vps215059
vps215481
vps214976
vps215767
vps222445
vps223817
vps215812
vps230491
vps230787
vps240057
vps252681
vps261299
vps262327
vps262746
vps263067
vps263173
vps262706
vps243278
vps228330
vps279064
vps286068
vps286117
vps286118
vps286155
vps234225
vps287877
vps288265
vps295764
vps295246
vps297366
vps297369
vps297379
vps301812
vps280551
vps214739
vps216450
vps217041
vps217174
vps229337
vps232287
vps235829
vps236805
vps238270
vps246602
vps248815
vps252039
vps255267
vps260269
vps236071
vps285303
vps294829
vps286256
vps208713
vps209492
vps221151
vps224794
vps224811
vps225977
vps225655
vps218836
vps230690
vps232489
vps234057
vps234288
vps234291
vps234537
vps234654
vps233958
vps238249
vps239607
vps241241
vps217369
vps258951
vps231900
vps259052
vps265416
vps214952
vps277147
vps278418
vps279782
vps281842
vps283767
vps286782
vps287542
vps287608
vps286613
vps249389
vps297283
vps302349
vps215497
vps215641
vps222785
vps224609
vps224843
vps225298
vps225471
vps226917
vps225304
vps229310
vps230602
vps231814
vps231715
vps232068
vps232348
vps232388
vps235652
vps236626
vps247459
vps247647
vps260701
vps266761
vps271574
vps279813
vps280583
vps227688
vps78220
vps226096
vps282639
vps295671
vps212061
vps215359
vps219971
vps220138
vps220228
vps229370
vps232403
vps232796
vps230243
vps233899
vps234590
vps237873
vps238362
vps247965
vps217131
vps283232
vps300255
vps11997
vps205821
vps215750
vps219713
vps220852
vps222305
vps226377
vps232613
vps236052
vps239879
vps240305
vps243657
vps248670
vps250456
vps92056
vps253774
vps254472
vps254741
vps266555
vps222805
vps231783
vps206908
vps207105
vps207898
vps208197
vps208235
vps208277
vps208932
vps209670
vps214997
vps229049
vps233973
vps234211
vps248551
vps226124
vps271111
vps278822
vps290231
vps293717
vps192856
vps192892
vps192987
vps193748
vps194508
vps194642
vps194916
vps194910
vps195857
vps194888
vps196104
vps196893
vps197820
vps197934
vps198476
vps198777
vps198094
vps197136
vps197550
vps193909
vps206426
vps206435
vps206444
vps206501
vps206544
vps206580
vps206591
vps206641
vps206667
vps206724
vps206694
vps206803
vps206811
vps206548
vps211570
vps211633
vps211762
vps211912
vps211922
vps211643
vps212055
vps212117
vps212121
vps212151
vps212189
vps211609
vps212226
vps212281
vps235085
vps226628
vps202443
vps204478
vps204462
vps208631
vps208753
vps217830
vps208585
vps203514
vps223150
vps223233
vps223975
vps223977
vps222922
vps226602
vps234072
vps234161
vps234358
vps234533
vps235492
vps231470
vps211667
vps237132
vps233966
vps243846
vps214624
vps259812
vps259807
vps260303
vps263561
vps265084
vps53637
vps270078
vps270311
vps272144
vps279513
vps279654
vps283449
vps297508
vps194967
vps197635
vps197771
vps198370
vps206383
vps206506
vps206547
vps206717
vps210807
vps211678
vps211748
vps212385
vps214299
vps214658
vps215901
vps216270
vps216385
vps217865
vps219153
vps225001
vps225837
vps227029
vps227901
vps227889
vps232259
vps236199
vps236391
vps216951
vps259898
vps261514
vps261739
vps260352
vps263403
vps275671
vps266941
vps279662
vps285744
vps288897
vps226351
vps296057
vps297099
vps288731
vps199396
vps200805
vps201854
vps207293
vps209426
vps214086
vps213963
vps216472
vps217391
vps217229
vps228142
vps224365
vps234505
vps234985
vps235632
vps235675
vps235677
vps235797
vps236210
vps236472
vps236604
vps225862
vps237156
vps237158
vps237349
vps236265
vps238816
vps239254
vps239466
vps239757
vps242983
vps243349
vps243783
vps245263
vps246417
vps252622
vps255552
vps255854
vps255883
vps220165
vps259009
vps261443
vps261893
vps262929
vps265604
vps258228
vps269144
vps274924
vps278001
vps279067
vps236444
vps283407
vps284532
vps278556
vps262828
vps287329
vps287712
vps292588
vps213838
vps295511
vps302045
vps302077
vps302575
vps217031
vps198657
vps203753
vps210749
vps212642
vps222244
vps205784
vps224862
vps227473
vps230158
vps231082
vps231555
vps236235
vps236324
vps236776
vps224450
vps240593
vps240735
vps245756
vps231673
vps250913
vps250773
vps251488
vps252101
vps253454
vps235865
vps258425
vps262272
vps264830
vps262767
vps272832
vps263659
vps282258
vps283872
vps286022
vps282896
vps298005
vps302210
vps226375
vps204097
vps204011
vps210573
vps211097
vps214093
vps215333
vps218145
vps218157
vps219099
vps220008
vps219977
vps220107
vps220146
vps220420
vps216387
vps220226
vps226085
vps228109
vps229439
vps229770
vps230249
vps230269
vps232036
vps232998
vps220693
vps220701
vps237770
vps238069
vps238147
vps238827
vps240145
vps243091
vps231036
vps246444
vps240022
vps226742
vps248437
vps251898
vps260173
vps260465
vps260702
vps262536
vps263409
vps266735
vps270687
vps267023
vps276594
vps282727
vps285539
vps285641
vps286216
vps286265
vps286398
vps286412
vps230471
vps284836
vps286354
vps290730
vps225515
vps292556
vps234983
vps293316
vps294199
vps293249
vps295693
vps297513
vps251756
vps296355
vps220168
vps261414
vps220698
vps228248
vps210157
vps279080
vps201430
vps203017
vps203667
vps203735
vps204410
vps205763
vps213532
vps220062
vps220952
vps221814
vps223096
vps225050
vps232506
vps232561
vps236487
vps236656
vps243527
vps244664
vps244962
vps245995
vps248282
vps252208
vps236315
vps259421
vps260688
vps278049
vps280951
vps301339
vps90075
vps203032
vps204238
vps208781
vps209970
vps210448
vps210271
vps213783
vps214156
vps214645
vps214684
vps214701
vps214840
vps215032
vps217716
vps218240
vps220311
vps220932
vps221805
vps223159
vps223190
vps224966
vps227108
vps229675
vps229677
vps230455
vps233485
vps234207
vps218182
vps230751
vps243548
vps219667
vps246761
vps247843
vps250448
vps252350
vps260313
vps261167
vps262253
vps262902
vps263169
vps263225
vps268041
vps274281
vps278014
vps218039
vps281501
vps282885
vps283448
vps283352
vps214428
vps287721
vps228138
vps290835
vps282732
vps294967
vps301801
vps301899
vps301933
vps302036
vps229316
vps224585
vps208810
vps208811
vps208815
vps208937
vps208221
vps208988
vps208925
vps209642
vps209686
vps209959
vps209985
vps218180
vps218224
vps218120
vps217900
vps219551
vps218986
vps222973
vps222977
vps239360
vps239951
vps242265
vps228402
vps255547
vps274343
vps274996
vps278799
vps285369
vps286958
vps286744
vps287448
vps287926
vps280610
vps286951
vps299294
vps296255
vps235404
vps207758
vps207854
vps209980
vps218445
vps218712
vps219420
vps222141
vps222005
vps221465
vps225338
vps224982
vps229302
vps222571
vps231555
vps232125
vps234682
vps238087
vps237701
vps107270
vps240106
vps243691
vps254689
vps260709
vps260925
vps262639
vps194730
vps265111
vps274545
vps281932
vps287436
vps222105
vps185554
vps297113
vps299862
vps301564
vps212050
vps291001
vps233474
vps195969
vps197677
vps211624
vps211887
vps215037
vps220928
vps228823
vps239674
vps259727
vps263062
vps263277
vps278973
vps280297
vps280640
vps204704
vps209489
vps210305
vps214740
vps216375
vps216707
vps217071
vps219665
vps220734
vps223204
vps229517
vps230344
vps229558
vps234500
vps234668
vps237056
vps247370
vps261667
vps222095
vps263588
vps273381
vps276098
vps276561
vps281197
vps285259
vps294445
vps295340
vps299037
vps296343
vps300182
vps201832
vps203164
vps211196
vps214212
vps222470
vps222443
vps225902
vps226080
vps226119
vps226150
vps226260
vps226316
vps226381
vps226388
vps226404
vps225816
vps221301
vps232010
vps233150
vps234864
vps239298
vps239868
vps241535
vps250477
vps259886
vps262033
vps271527
vps275410
vps279302
vps279718
vps279835
vps283414
vps284409
vps284550
vps284632
vps286938
vps287894
vps293963
vps210286
vps214482
vps215053
vps215741
vps215910
vps225630
vps231370
vps236636
vps240111
vps241554
vps259989
vps260207
vps260451
vps260618
vps261001
vps257516
vps279902
vps282783
vps148895
vps204691
vps217806
vps218465
vps218914
vps223841
vps228904
vps229336
vps221293
vps240363
vps233511
vps249882
vps262899
vps279521
vps282956
vps288074
vps289992
vps288654
vps296169
vps206682
vps211900
vps228206
vps239229
vps251524
vps252277
vps263656
vps272750
vps272482
vps278208
vps278585
vps280539
vps279819
vps281616
vps237793
vps238062
vps292136
vps297031
vps207545
vps214999
vps217072
vps219790
vps220709
vps220841
vps221707
vps224297
vps225596
vps232799
vps218596
vps237064
vps237502
vps238536
vps238805
vps244133
vps231487
vps204447
vps232458
vps227489
vps262813
vps263475
vps276813
vps277148
vps278426
vps278568
vps281067
vps281549
vps281781
vps281807
vps283400
vps281612
vps284124
vps284230
vps285657
vps290717
vps297785
vps238014
vps199206
vps199295
vps201875
vps206751
vps207890
vps209119
vps209731
vps211652
vps211783
vps212374
vps212930
vps213069
vps213978
vps214660
vps216676
vps218198
vps221178
vps221845
vps224059
vps225555
vps226084
vps226224
vps226861
vps230482
vps231419
vps232238
vps231547
vps233003
vps234811
vps235351
vps235867
vps237536
vps237724
vps237857
vps236736
vps238283
vps239147
vps239352
vps211378
vps242067
vps242411
vps255747
vps260972
vps263644
vps278992
vps279323
vps279854
vps280534
vps284961
vps286943
vps286949
vps207337
vps258202
vps301207
vps302044
vps210524
vps285892
vps202646
vps203201
vps203885
vps204666
vps208463
vps209966
vps210924
vps210971
vps211338
vps212774
vps213799
vps216272
vps216365
vps216806
vps217845
vps218916
vps219171
vps220939
vps220938
vps213907
vps224497
vps225524
vps230033
vps232338
vps232902
vps230231
vps238250
vps248525
vps212149
vps189224
vps256816
vps259720
vps259909
vps260777
vps260780
vps261166
vps210264
vps204578
vps270836
vps204079
vps272304
vps205179
vps277185
vps277694
vps278552
vps278787
vps277431
vps279014
vps279496
vps280656
vps280837
vps281369
vps281383
vps286031
vps286957
vps238227
vps287563
vps288159
vps252306
vps291457
vps295729
vps295799
vps296080
vps298669
vps213087
vps215794
vps216102
vps216155
vps214779
vps219027
vps220326
vps220443
vps220498
vps220771
vps222595
vps221450
vps226745
vps226894
vps227229
vps222199
vps227815
vps227841
vps228659
vps228989
vps229182
vps229833
vps229533
vps231983
vps235493
vps227218
vps237543
vps237578
vps237730
vps238261
vps238092
vps240252
vps231186
vps245495
vps220009
vps252813
vps256044
vps256341
vps259956
vps260106
vps260979
vps261324
vps261386
vps261883
vps269940
vps283000
vps283692
vps254711
vps287916
vps288175
vps287572
vps289401
vps289636
vps281456
vps293882
vps264742
vps294283
vps295722
vps297505
vps297801
vps296593
vps302226
vps296490
vps241272
vps192915
vps195874
vps198414
vps206438
vps211726
vps212225
vps212448
vps199906
vps205327
vps207303
vps207329
vps208147
vps209559
vps209946
vps213264
vps213290
vps213770
vps213867
vps212835
vps217961
vps218168
vps218433
vps218468
vps40346
vps218676
vps220261
vps220369
vps220454
vps221211
vps221834
vps221161
vps223141
vps223273
vps224308
vps215878
vps226082
vps226394
vps218668
vps229069
vps231761
vps231792
vps231795
vps232118
vps232616
vps233017
vps233165
vps220694
vps220695
vps216540
vps234375
vps234527
vps238184
vps213318
vps231805
vps243342
vps222149
vps224783
vps246424
vps207030
vps260900
vps262206
vps262640
vps260346
vps254939
vps265677
vps270657
vps238721
vps277187
vps279220
vps281179
vps282041
vps283758
vps217768
vps285924
vps286563
vps240277
vps291245
vps297084
vps297290
vps297645
vps282835
vps302428
vps302530
vps205209
vps210190
vps204996
vps210414
vps215068
vps214983
vps218756
vps218767
vps218770
vps218820
vps218747
vps218866
vps219072
vps219055
vps219109
vps219127
vps219112
vps219161
vps219321
vps219588
vps219628
vps219648
vps219756
vps219766
vps219770
vps219811
vps219649
vps218111
vps219710
vps219939
vps220021
vps220098
vps211808
vps223848
vps224460
vps223839
vps229764
vps230846
vps219062
vps226223
vps239768
vps240339
vps242790
vps245017
vps246287
vps246288
vps245277
vps250758
vps218730
vps256527
vps257684
vps260183
vps265986
vps273911
vps276112
vps233221
vps278525
vps279135
vps280257
vps282199
vps282651
vps277358
vps281821
vps289077
vps288762
vps231066
vps294643
vps295174
vps295382
vps295952
vps216279
vps221550
vps287207
vps216619
vps199597
vps205229
vps205507
vps205603
vps206019
vps206347
vps210580
vps213397
vps214094
vps214157
vps214787
vps214830
vps214296
vps215124
vps215206
vps215207
vps215408
vps215513
vps215520
vps215739
vps212588
vps214878
vps223207
vps228723
vps229875
vps229985
vps233803
vps240656
vps241153
vps241275
vps242999
vps243095
vps246009
vps246224
vps246192
vps246247
vps250139
vps221116
vps257508
vps229214
vps260525
vps233663
vps262001
vps262291
vps262330
vps263099
vps263105
vps263527
vps263326
vps264261
vps269178
vps269891
vps275093
vps277490
vps278448
vps266644
vps279525
vps284920
vps286836
vps297220
vps298094
vps294830
vps214706
vps218184
vps218862
vps225518
vps226809
vps222468
vps231807
vps233063
vps235992
vps240685
vps241378
vps259730
vps261607
vps277847
vps295935
vps296385
vps301088
vps236554
vps213411
vps215148
vps206705
vps226441
vps214498
vps230214
vps230216
vps230534
vps228741
vps231347
vps236896
vps240673
vps260953
vps260562
vps263457
vps268904
vps233624
vps278143
vps286102
vps298897
vps205339
vps205745
vps207264
vps214560
vps214868
vps216066
vps216837
vps218375
vps218842
vps223214
vps224201
vps227624
vps227832
vps227963
vps229846
vps229909
vps230152
vps230169
vps232004
vps231791
vps232069
vps232488
vps233260
vps231707
vps238112
vps239290
vps240084
vps246453
vps268069
vps278379
vps283396
vps283451
vps283803
vps216453
vps281935
vps237634
vps297374
vps297377
vps207147
vps208422
vps208625
vps208065
vps208824
vps213994
vps212996
vps223220
vps224595
vps225332
vps226064
vps225311
vps225066
vps233015
vps207520
vps262811
vps202648
vps214067
vps217221
vps209062
vps222140
vps223184
vps225454
vps227467
vps223034
vps234279
vps234335
vps234589
vps235041
vps235718
vps235814
vps236499
vps236579
vps237269
vps232515
vps232318
vps238448
vps239020
vps239096
vps240521
vps242473
vps243807
vps244160
vps247207
vps250475
vps256495
vps257304
vps259739
vps262075
vps262807
vps265490
vps263593
vps266022
vps220826
vps266584
vps268810
vps273048
vps272665
vps272320
vps278115
vps278428
vps278452
vps279488
vps280696
vps282435
vps281504
vps283326
vps283368
vps283722
vps284866
vps288652
vps292371
vps286833
vps296134
vps298007
vps230742
vps301806
vps207951
vps196374
vps196464
vps211565
vps212133
vps212366
vps214729
vps211751
vps221651
vps225746
vps234159
vps236088
vps259400
vps288543
vps229847
vps207790
vps210097
vps211287
vps211474
vps211844
vps221015
vps221409
vps221417
vps221445
vps221877
vps222903
vps225165
vps226281
vps226689
vps226710
vps220593
vps233604
vps234612
vps233049
vps239242
vps240369
vps223924
vps218775
vps264788
vps276541
vps284567
vps285760
vps286659
vps286676
vps286740
vps286789
vps284636
vps290439
vps280881
vps296453
vps301714
vps204595
vps222529
vps226807
vps226041
vps219213
vps226106
vps233533
vps239645
vps249569
vps254867
vps260524
vps262853
vps261425
vps274939
vps286430
vps286898
vps290904
vps192622
vps192917
vps193558
vps194973
vps197495
vps206375
vps206593
vps211601
vps211940
vps212638
vps200874
vps207962
vps217129
vps230576
vps238073
vps239245
vps237561
vps239511
vps250504
vps281401
vps284432
vps301437
vps203275
vps217735
vps229756
vps230377
vps236316
vps239164
vps238987
vps245566
vps211955
vps255880
vps230150
vps296430
vps296944
vps216627
vps195745
vps214155
vps216434
vps216517
vps220148
vps223081
vps217049
vps224976
vps225802
vps226336
vps227462
vps228057
vps225891
vps231903
vps237809
vps237880
vps247739
vps251387
vps251978
vps253720
vps253738
vps255553
vps258534
vps258705
vps259816
vps259939
vps259966
vps260464
vps260579
vps271526
vps286367
vps286748
vps287052
vps205724
vps297241
vps301312
vps232033
vps302019
vps210336
vps215959
vps217871
vps211836
vps219092
vps219156
vps220804
vps220783
vps221153
vps222752
vps224993
vps228679
vps238353
vps243632
vps240345
vps281989
vps286429
vps297018
vps206398
vps207556
vps208889
vps209461
vps213174
vps213768
vps217004
vps218031
vps218087
vps219007
vps210468
vps220145
vps220336
vps220936
vps223591
vps224058
vps224527
vps224906
vps225583
vps225554
vps239676
vps245074
vps245442
vps226095
vps247515
vps249707
vps226632
vps250522
vps251989
vps255439
vps258577
vps258598
vps259590
vps260449
vps260450
vps262692
vps267069
vps267070
vps279368
vps279411
vps279904
vps284661
vps285570
vps285870
vps286060
vps286150
vps286347
vps287502
vps288188
vps241615
vps295698
vps297352
vps212852
vps213789
vps217616
vps218045
vps218528
vps218830
vps219860
vps219426
vps224462
vps244341
vps267085
vps217906
vps280752
vps281102
vps281418
vps194752
vps207330
vps214091
vps220370
vps220811
vps222974
vps227915
vps235088
vps235376
vps233748
vps248288
vps250390
vps250811
vps260440
vps260459
vps239713
vps260912
vps263190
vps263635
vps264145
vps269896
vps270054
vps275279
vps285099
vps285128
vps285687
vps285712
vps285827
vps225251
vps253699
vps278311
vps294521
vps295916
vps296212
vps300613
vps228706
vps203845
vps208806
vps209251
vps211037
vps216132
vps216264
vps217325
vps220802
vps219674
vps227645
vps227690
vps227912
vps228126
vps228304
vps228361
vps228450
vps228514
vps228535
vps228257
vps229285
vps232283
vps223149
vps237889
vps240016
vps240018
vps240179
vps249824
vps257982
vps263592
vps276598
vps288591
vps295551
vps205404
vps210334
vps214808
vps217020
vps220800
vps220998
vps230593
vps231523
vps237142
vps238333
vps240479
vps240532
vps240711
vps240730
vps240797
vps243573
vps223974
vps245080
vps220125
vps250801
vps251475
vps252313
vps252689
vps255543
vps260441
vps210626
vps263269
vps240480
vps270686
vps270694
vps270695
vps270709
vps270784
vps270545
vps271691
vps271809
vps272067
vps273953
vps273954
vps274017
vps277812
vps261034
vps278600
vps278770
vps278881
vps278897
vps278910
vps234241
vps281816
vps281860
vps214429
vps279202
vps283023
vps283167
vps284083
vps283091
vps286041
vps244707
vps296267
vps300251
vps301751
vps301759
vps285706
vps202776
vps217912
vps218106
vps222197
vps230666
vps231781
vps234490
vps235868
vps235864
vps236758
vps236777
vps237899
vps240734
vps243667
vps246952
vps254800
vps259285
vps259806
vps261159
vps261409
vps264017
vps269182
vps268101
vps278393
vps278659
vps279137
vps279856
vps282413
vps284010
vps284049
vps288048
vps296076
vps297303
vps285996
vps297376
vps269040
vps220901
vps291652
vps200395
vps204187
vps205560
vps210474
vps215935
vps217603
vps218625
vps221754
vps213410
vps226774
vps226841
vps226915
vps227230
vps227291
vps227868
vps227881
vps227885
vps227987
vps228046
vps228359
vps228429
vps228507
vps228557
vps228715
vps228737
vps228777
vps229024
vps228503
vps229181
vps225948
vps228880
vps228902
vps230405
vps220839
vps228917
vps220692
vps236171
vps236524
vps240123
vps240220
vps242102
vps217969
vps256095
vps279063
vps281196
vps283355
vps254016
vps286179
vps287917
vps287918
vps287957
vps288117
vps288395
vps288660
vps290797
vps291074
vps285652
vps292759
vps295556
vps296238
vps298670
vps300368
vps301110
vps302257
vps208555
vps209083
vps215577
vps217795
vps217836
vps218683
vps222600
vps222827
vps223161
vps221563
vps225976
vps230248
vps232862
vps233008
vps236092
vps237421
vps236101
vps254848
vps260463
vps52854
vps241083
vps270271
vps279811
vps227556
vps281776
vps281825
vps282793
vps282805
vps282947
vps252135
vps283273
vps283382
vps283392
vps283453
vps283482
vps283629
vps283153
vps280935
vps283893
vps283898
vps284134
vps284203
vps284516
vps284655
vps284656
vps284657
vps284658
vps284600
vps287162
vps283452
vps293192
vps218509
vps295966
vps297879
vps297502
vps298170
vps299556
vps301813
vps301817
vps301766
vps302121
vps302425
vps234938
vps229479
vps197345
vps199500
vps206216
vps208833
vps210343
vps210620
vps219815
vps219896
vps220132
vps222920
vps225856
vps227030
vps229873
vps240346
vps240501
vps240684
vps242869
vps249050
vps260853
vps260949
vps261058
vps263321
vps263349
vps272997
vps281775
vps287855
vps289718
vps290054
vps293062
vps296260
vps137148
vps213340
vps218092
vps220353
vps220655
vps225152
vps231779
vps235089
vps236200
vps240203
vps240715
vps222163
vps255096
vps258120
vps262404
vps263114
vps263677
vps264144
vps264146
vps264236
vps265096
vps235121
vps268820
vps270264
vps269053
vps282359
vps203867
vps284317
vps289892
vps297108
vps203681
vps297364
vps302300
vps198395
vps193540
vps206873
vps207484
vps212205
vps217542
vps221987
vps224174
vps212940
vps251721
vps279237
vps281029
vps288193
vps285402
vps294982
vps238795
vps206140
vps207767
vps214323
vps214352
vps214353
vps214555
vps219309
vps222631
vps223783
vps224434
vps229952
vps231637
vps233513
vps233727
vps234469
vps221009
vps239438
vps228527
vps240251
vps240358
vps252668
vps255123
vps259671
vps259697
vps268046
vps275174
vps257909
vps278012
vps278013
vps278015
vps281373
vps281390
vps281557
vps281591
vps281931
vps240349
vps283774
vps272877
vps285285
vps239174
vps289733
vps296923
vps297020
vps297041
vps297579
vps282225
vps298456
vps281449
vps252854
vps264711
vps201581
vps206042
vps208509
vps208970
vps210944
vps211295
vps223063
vps227809
vps231988
vps233906
vps238416
vps260052
vps265917
vps260256
vps280827
vps281309
vps281493
vps209932
vps201852
vps204508
vps204560
vps204837
vps204882
vps205213
vps205913
vps206238
vps205843
vps210247
vps210528
vps213960
vps214152
vps214794
vps215118
vps215395
vps215098
vps215490
vps215612
vps215740
vps215822
vps215992
vps216148
vps216226
vps218520
vps192791
vps219973
vps218783
vps218424
vps226571
vps229283
vps229289
vps229951
vps222144
vps233074
vps233412
vps178108
vps238841
vps241805
vps241440
vps246157
vps246622
vps248784
vps249287
vps255069
vps256291
vps258451
vps260593
vps261069
vps261335
vps262427
vps262538
vps260808
vps263152
vps272753
vps279878
vps234076
vps280778
vps285447
vps285673
vps285906
vps287215
vps287419
vps287659
vps225666
vps238388
vps288865
vps293623
vps101838
vps214665
vps298092
vps241552
vps295162
vps212953
vps217764
vps217741
vps217842
vps218066
vps218148
vps224321
vps226054
vps226172
vps230790
vps217778
vps240851
vps220700
vps281777
vps211401
vps213818
vps214571
vps214775
vps220388
vps230345
vps230698
vps233545
vps240291
vps249698
vps237426
vps257454
vps261226
vps266636
vps277325
vps278008
vps279289
vps297025
vps220170
vps231015
vps204043
vps206228
vps208127
vps214158
vps214537
vps212576
vps214977
vps216228
vps216364
vps214695
vps216735
vps218838
vps219937
vps219792
vps220312
vps215816
vps230603
vps234652
vps257292
vps263077
vps262773
vps272066
vps276936
vps281452
vps281863
vps285944
vps274602
vps295407
vps297367
vps297370
vps302114
vps285671
vps272860
vps289976
vps212660
vps213751
vps213701
vps211719
vps226685
vps234082
vps237863
vps241089
vps262770
vps275412
vps234282
vps283748
vps285314
vps286345
vps286819
vps286932
vps297361
vps222013
vps234648
vps236180
vps236066
vps212213
vps260246
vps261031
vps270838
vps278569
vps278861
vps279736
vps279831
vps279847
vps230739
vps225684
vps294831
vps294832
vps297136
vps295473
vps199046
vps205095
vps206325
vps216555
vps220006
vps221643
vps222362
vps234623
vps235515
vps236309
vps235873
vps241608
vps246326
vps261869
vps267954
vps289982
vps289938
vps290963
vps301112
vps193981
vps200522
vps203651
vps207926
vps214420
vps215714
vps215717
vps215718
vps215781
vps215608
vps217035
vps218126
vps219468
vps220158
vps221126
vps221355
vps221950
vps224004
vps224154
vps222813
vps230428
vps230601
vps232667
vps237101
vps222248
vps239500
vps256762
vps263671
vps212156
vps285098
vps291081
vps295489
vps293752
vps298024
vps298974
vps301140
vps301496
vps220545
vps213567
vps218147
vps223300
vps223547
vps223893
vps217777
vps238928
vps245623
vps254990
vps226679
vps255706
vps217819
vps278832
vps279862
vps280570
vps285312
vps278409
vps301081
vps294140
vps203974
vps215094
vps215028
vps223286
vps223788
vps225965
vps218115
vps235378
vps235686
vps237690
vps243277
vps248983
vps214889
vps259873
vps261410
vps262819
vps262959
vps264578
vps268549
vps233659
vps272573
vps273092
vps278002
vps278467
vps278848
vps278908
vps279312
vps279435
vps279972
vps280085
vps280483
vps280558
vps280568
vps279859
vps278957
vps219022
vps298101
vps297378
vps280335
vps204678
vps205176
vps207595
vps222237
vps222638
vps225693
vps226730
vps228474
vps221778
vps234903
vps236675
vps241640
vps265918
vps279327
vps279834
vps287881
vps294438
vps302031
vps207888
vps207948
vps207974
vps208042
vps207964
vps206941
vps203366
vps218091
vps220722
vps220857
vps220799
vps221134
vps221341
vps222542
vps227024
vps229470
vps208813
vps232407
vps233153
vps234101
vps235891
vps240475
vps240490
vps220990
vps253582
vps259793
vps261741
vps263669
vps199789
vps238081
vps221356
vps221162
vps280920
vps283766
vps289449
vps291015
vps228145
vps231850
vps203130
vps205017
vps206181
vps208424
vps210570
vps218451
vps228446
vps235392
vps240100
vps242011
vps247509
vps248787
vps282186
vps285589
vps286617
vps287400
vps287960
vps287885
vps297510
vps291941
vps297829
vps203251
vps214822
vps283358
vps283778
vps284667
vps285339
vps239679
vps285545
vps286935
vps287797
vps283941
vps283943
vps283942
vps297277
vps298436
vps298437
vps204259
vps209525
vps209665
vps212878
vps213958
vps214562
vps213086
vps216710
vps216816
vps214046
vps221114
vps225409
vps232925
vps227126
vps240510
vps215894
vps275702
vps276106
vps206096
vps282581
vps283144
vps283474
vps283627
vps285127
vps285151
vps285304
vps285575
vps286318
vps274476
vps262961
vps216243
vps216311
vps216726
vps217089
vps219699
vps220108
vps220287
vps220499
vps220592
vps223818
vps233885
vps238768
vps238770
vps241321
vps243225
vps249955
vps250101
vps253960
vps253443
vps215931
vps259903
vps215523
vps295900
vps281659
vps214781
vps216841
vps222975
vps233038
vps233788
vps240065
vps252140
vps257089
vps257162
vps276562
vps278651
vps279276
vps283410
vps287496
vps301614
vps294325
vps226395
vps279315
vps206134
vps208861
vps208809
vps208892
vps209890
vps209896
vps218516
vps218882
vps219137
vps222817
vps222864
vps222969
vps222971
vps222226
vps224690
vps226675
vps227996
vps229819
vps229129
vps234430
vps240121
vps243006
vps210144
vps252663
vps259883
vps262177
vps263605
vps272392
vps286584
vps286647
vps286615
vps214222
vps223659
vps228837
vps229662
vps229664
vps229674
vps234602
vps235210
vps236480
vps236280
vps236563
vps195066
vps236806
vps249667
vps263610
vps268045
vps246475
vps284425
vps284625
vps284627
vps284628
vps280780
vps286798
vps287080
vps294510
vps296062
vps301176
vps287453
vps204397
vps209821
vps212962
vps213591
vps218386
vps222550
vps222556
vps224559
vps224561
vps224771
vps224911
vps224984
vps225191
vps224989
vps225154
vps226993
vps227015
vps228194
vps228580
vps229437
vps235397
vps236308
vps236249
vps236525
vps236670
vps236958
vps231300
vps237067
vps237265
vps238640
vps240279
vps243222
vps243546
vps244310
vps225373
vps249899
vps241332
vps251509
vps251887
vps255059
vps260101
vps264404
vps266662
vps273913
vps209632
vps276180
vps263373
vps276496
vps278081
vps278338
vps275705
vps279208
vps281470
vps282119
vps282254
vps238273
vps285147
vps285341
vps285583
vps286077
vps219822
vps264600
vps287269
vps286066
vps288815
vps294435
vps296049
vps298067
vps245944
vps301125
vps301910
vps302455
vps292785
vps302009
vps284622
vps208994
vps218552
vps221227
vps221810
vps224489
vps233386
vps234390
vps234535
vps226930
vps285608
vps300783
vps204520
vps205758
vps208656
vps210145
vps210344
vps214602
vps214786
vps215188
vps215389
vps216178
vps216482
vps218731
vps221766
vps225417
vps231068
vps231381
vps233903
vps235854
vps238272
vps238518
vps210089
vps255834
vps262971
vps230406
vps227953
vps282147
vps282828
vps282120
vps296298
vps296344
vps301909
vps219797
vps220494
vps220524
vps221357
vps224634
vps224688
vps226146
vps226531
vps236121
vps238206
vps250165
vps261305
vps261317
vps257515
vps278815
vps258826
vps286044
vps298095
vps201636
vps202132
vps202544
vps202639
vps202604
vps204695
vps205892
vps205954
vps213778
vps214058
vps214373
vps214442
vps214543
vps214885
vps216104
vps216558
vps216807
vps216514
vps219800
vps220154
vps238495
vps245679
vps234845
vps257649
vps260992
vps224718
vps276024
vps201816
vps287821
vps297662
vps298002
vps208522
vps282181
vps282377
vps208915
vps214493
vps214921
vps216361
vps227989
vps228198
vps228331
vps228443
vps228654
vps231318
vps234624
vps234622
vps260188
vps260821
vps260130
vps233812
vps279526
vps290955
vps205606
vps215810
vps217258
vps218863
vps219990
vps220959
vps223733
vps224755
vps225834
vps231904
vps232137
vps245922
vps264293
vps265739
vps269776
vps288734
vps287390
vps292253
vps231415
vps201275
vps207858
vps207978
vps215638
vps216370
vps221000
vps221035
vps221472
vps222162
vps222599
vps222935
vps225949
vps225950
vps226284
vps226507
vps226575
vps226364
vps227781
vps230404
vps230830
vps231486
vps232115
vps233402
vps233652
vps234062
vps233985
vps235288
vps235360
vps235602
vps236938
vps220955
vps237903
vps241096
vps245101
vps252009
vps193980
vps239110
vps260965
vps260995
vps262382
vps264825
vps269949
vps218350
vps272542
vps279431
vps279765
vps280800
vps281548
vps282287
vps289635
vps280233
vps300588
vps200366
vps203125
vps204102
vps208949
vps241923
vps273548
vps276053
vps276217
vps278093
vps281171
vps93605
vps281409
vps281524
vps269349
vps281861
vps239452
vps214912
vps210972
vps227460
vps233544
vps237301
vps251980
vps263448
vps266603
vps276521
vps279372
vps281211
vps281233
vps281326
vps281389
vps281408
vps281472
vps281613
vps281614
vps281623
vps281754
vps282743
vps207095
vps283007
vps284007
vps284014
vps284529
vps285036
vps283538
vps281689
vps253912
vps287884
vps279425
vps83700
vps213122
vps214225
vps221561
vps229840
vps226666
vps230715
vps232860
vps231832
vps235869
vps238723
vps197579
vps240538
vps246995
vps252623
vps255591
vps240518
vps269723
vps234731
vps202133
vps203801
vps204523
vps204376
vps202603
vps206095
vps209416
vps204570
vps214259
vps217190
vps219291
vps220308
vps222079
vps254280
vps282760
vps283183
vps298440
vps205714
vps206184
vps213065
vps226520
vps235805
vps238003
vps238017
vps238527
vps238724
vps238936
vps238952
vps239122
vps225007
vps233902
vps280378
vps284796
vps285133
vps293775
vps206692
vps220384
vps223251
vps231508
vps239623
vps249948
vps275794
vps256797
vps276119
vps278022
vps295336
vps302514
vps194434
vps198445
vps206427
vps206741
vps206697
vps211980
vps212057
vps212095
vps212181
vps212317
vps227032
vps227892
vps227925
vps227971
vps227986
vps228176
vps228449
vps228558
vps228897
vps228984
vps228038
vps228890
vps241040
vps241336
vps237088
vps283581
vps287959
vps287503
vps296730
vps205712
vps206167
vps215997
vps221342
vps221857
vps197632
vps222032
vps222165
vps221210
vps13611
vps229180
vps236785
vps236623
vps247830
vps266319
vps207018
vps233082
vps287678
vps301725
vps213993
vps218366
vps220365
vps220895
vps222322
vps143406
vps221936
vps228041
vps228739
vps231562
vps236400
vps240158
vps242870
vps252894
vps255972
vps260729
vps263239
vps269314
vps274970
vps277221
vps278768
vps279130
vps280911
vps286141
vps287824
vps295979
vps299008
vps301558
vps301152
vps260957
vps206624
vps221801
vps222021
vps224802
vps212079
vps233755
vps235053
vps235359
vps236610
vps236159
vps236500
vps222569
vps237764
vps239651
vps240130
vps234660
vps240388
vps243996
vps238798
vps262597
vps265085
vps271297
vps272264
vps280794
vps287363
vps286747
vps287787
vps288548
vps288592
vps288630
vps287121
vps288557
vps204378
vps210349
vps210367
vps208224
vps218151
vps225461
vps227197
vps224414
vps228158
vps236206
vps238431
vps240773
vps273775
vps275036
vps275700
vps278092
vps278017
vps297320
vps213739
vps224408
vps224991
vps251519
vps255266
vps259215
vps179194
vps261191
vps281956
vps283926
vps286428
vps286464
vps287032
vps286481
vps286918
vps298794
vps298438
vps297360
vps202653
vps209272
vps209863
vps216225
vps220831
vps221703
vps222109
vps222161
vps222910
vps223065
vps223142
vps223880
vps224631
vps225546
vps226307
vps229832
vps230719
vps232307
vps232619
vps234757
vps235063
vps235086
vps235090
vps235416
vps235418
vps212889
vps236508
vps237755
vps238080
vps238975
vps239520
vps237283
vps245755
vps248611
vps261098
vps262629
vps262738
vps264257
vps260983
vps262762
vps271689
vps226076
vps240332
vps278057
vps278232
vps278294
vps281406
vps282441
vps283016
vps282992
vps283192
vps285995
vps229797
vps286067
vps291087
vps295601
vps298006
vps204178
vps210989
vps215005
vps220419
vps223806
vps226003
vps226055
vps226286
vps225617
vps228899
vps229660
vps229661
vps229673
vps230926
vps231092
vps233112
vps234522
vps236176
vps237298
vps238050
vps240151
vps236481
vps229822
vps243134
vps243731
vps239670
vps235833
vps258628
vps266980
vps268042
vps268774
vps278016
vps278488
vps281806
vps282459
vps283788
vps283999
vps284511
vps285088
vps285478
vps285569
vps285707
vps287366
vps287862
vps284677
vps287425
vps291283
vps297355
vps237781
vps230851
vps212965
vps218798
vps218821
vps218899
vps219002
vps219026
vps219479
vps220022
vps220429
vps220413
vps220663
vps220719
vps220849
vps220795
vps219069
vps221544
vps222036
vps223112
vps223296
vps224950
vps225246
vps225282
vps226738
vps219296
vps230899
vps232154
vps235377
vps236139
vps220696
vps220697
vps220699
vps235206
vps244779
vps251074
vps225310
vps259738
vps259996
vps231203
vps277071
vps282572
vps266314
vps286359
vps231888
vps296350
vps297313
vps224799
vps216496
vps282358
vps241335
vps223042
vps302427
vps275337
vps204869
vps213881
vps214396
vps226948
vps227142
vps227943
vps224868
vps230566
vps256556
vps260453
vps262351
vps285153
vps297507
vps302188
vps302371
vps214591
vps258823
vps284246
vps192866
vps193414
vps193694
vps194149
vps195613
vps195765
vps195852
vps196473
vps197246
vps197903
vps197951
vps206823
vps211735
vps211870
vps211982
vps212789
vps212909
vps218415
vps224224
vps224323
vps227646
vps224591
vps230533
vps239941
vps241944
vps242730
vps260532
vps272834
vps278975
vps280917
vps284752
vps295973
vps282295
vps194490
vps203142
vps203622
vps204517
vps204698
vps205881
vps205986
vps207480
vps208434
vps209108
vps211625
vps212125
vps219695
vps218349
vps220360
vps220933
vps220934
vps220935
vps221848
vps226210
vps227773
vps228511
vps229663
vps229676
vps230638
vps236257
vps240401
vps258219
vps261169
vps269166
vps269464
vps269537
vps280492
vps278828
vps281877
vps279432
vps223534
vps285086
vps262090
vps289974
vps218347
vps297586
vps301589
vps235679
vps205008
vps208589
vps210202
vps210408
vps213196
vps218661
vps220473
vps221224
vps221225
vps218046
vps223964
vps226916
vps226965
vps229290
vps229953
vps231521
vps231798
vps232942
vps233316
vps233529
vps233760
vps234085
vps234274
vps234544
vps238413
vps238507
vps238698
vps245572
vps216832
vps238763
vps261354
vps261400
vps261515
vps261984
vps261985
vps245497
vps262537
vps262741
vps261388
vps261708
vps263175
vps262802
vps263134
vps268886
vps205678
vps273026
vps239820
vps278402
vps278960
vps279243
vps281244
vps281464
vps279203
vps281805
vps281955
vps282366
vps282372
vps282876
vps282958
vps282613
vps236319
vps284499
vps119780
vps288379
vps288380
vps282163
vps295760
vps297503
vps296224
vps280956
vps125918
vps300773
vps300948
vps302074
vps283207
vps203259
vps203583
vps202425
vps205283
vps205636
vps211195
vps213272
vps222189
vps222253
vps222254
vps227508
vps232255
vps232684
vps233071
vps236141
vps236613
vps236734
vps243554
vps245016
vps245158
vps245206
vps245383
vps246407
vps260818
vps260866
vps262698
vps263023
vps257700
vps278141
vps278161
vps284680
vps284859
vps285514
vps220058
vps288752
vps294519
vps155230
vps286410


Comment by OVH - Monday, 03 October 2016, 17:51PM

We have lost one OSD, and for an unknown reason all the cluster has slowed down. The Ceph team is still working on it.

No ETA for the moment.


Comment by OVH - Monday, 03 October 2016, 17:51PM

Our team is still working on it, the cluster is still locked, we are trying to unlock it.


Comment by OVH - Monday, 03 October 2016, 17:55PM

We are still trying do debug the ceph cluster. Some hexadump has been done without succes for the moment.
In parallel we are trying to restore some backup


Comment by OVH - Monday, 03 October 2016, 17:57PM

Summarizing: we had 1 failing HDD. After removing OSD from it, 17 objects were not found in Ceph after recovery. 7 of them are in 6 PGs which we cannot query or tell them to lose the objects. We have tried to manually force operations but have not succeeded. We are still looking for a solution to unblock those PGs.


Comment by OVH - Monday, 03 October 2016, 17:57PM

Good evening,
We have a technical issue on 1 Ceph cluster. We have about 200
harddisk in this cluster. Each 2TB. We have 3 copy of each data
on 3 disks. This cluster manage 120To of data.

1 of the disks was broken and we removed it. For some reasons,
Ceph stopped to working : 17 objectfs are missed. It should not.

The teams has been working on the issue from 6 :30am. We’ve tried
lot of things to get it up. Some members of the team have 11 hours
uptime, that is why we’ve decided to stop working during 1 hour.
We will restart working at 7pm with a meeting between the team in
Wroclaw (Poland), Roubaix (France) and Brest (France). The goal
is to find out an action plan to resolv the issue.

We know the impact is important. About 5000 VPS are using this
Ceph cluster. The deal is simple : it has to work even if we lose 66.66%
of the hosts. Here we lost 1 hard disk and it’s broken. Once the data
are UP, we will write the post-mortem and see if we can find out an
another technology for the block storage. Right now, the goal is to
restore the data and we will.

We are very sorry about this long downtime. Be sure, we are working
on this issue and we will resolv this problem. I can’t give you right now
the ETA.

Regards,
Octave


Comment by OVH - Monday, 03 October 2016, 17:58PM

From War Room.

Good evening
We recommand not to reboot your VPS.

The Ceph cluster is based on 24 servers. each with 12 disks. We have
the issue on 6 servers, not 24. That is why, not all 5000 VPS are down.
Some of them are down. The others continue to work with the remains
18 servers. We have an issue on 6 Placement Group (PG) and we have
10533 PG in this cluster. A small part of the data are fail but it can
impact lot of VPS. We are trying to estimate how many VPS are really
impacting, but it depends if 1) the VPS is using the 6 PG 2) the VPS wants
to read/write the data that are in the 6 PG.

The main issue is the version of the 17 objects. The objects are in the
version 696135'83055746. The version in the Ceph’s metadata is
696135'83055747. So Ceph doesn’t want to start. We’ve forced to
forget the bad files but it doesn’t work. Ceph is freezing.

We think that Ceph was trying to write the data on the failed disk.
Ceph has done it on the failed disk and updated the metadata, but
Ceph hasn’t done it on the others disks. Probably a bug in the version
we run 0.94.4. The last version is 0.94.9

4 action plans:

1)
we are patching the tool that can import/export the objects to force
the version of the object to 696135'83055747. 1 team is working on
this

2)
we are looking how to restart Ceph wihout 17 objects. an another
team is working on that.

3)
in the case, nothing works, we will start the Ceph without 6 PG, but
it means lost of some data. we don’t know which data would bel ost.
that is why we lauch a local backup of the data in the case we have
to work on it and restore the lost data in the futur. it’s a strategy
the « last case ». 1 team is working on that.

4)
we’are preapring the recovery of the data, but we are talking about
120TB. it will be slow, and the backup has 24 hours. 1 team is working
on that

Regards,
Octave


Comment by OVH - Monday, 03 October 2016, 17:58PM

The solution 2) begins to work

4) We are trying to find out the VPS that are down to restore
the data.


Comment by OVH - Monday, 03 October 2016, 17:59PM

2) we have only 1 objet that avoids to restart the Ceph.

we lose 17 objets with 4/8MB each. so 100MB/120TB. 0.8%


Comment by OVH - Monday, 03 October 2016, 17:59PM

2)

in Ceph, there is a command to forget 1 objet. it's worked
on 10 objets but it doesn't on the others 7 objectss.
we've found out that if we run the command to forget the
objet with "whie true" loop and in the same time we run
the command to restart ceph in the "while true" loop too
after a while, ceph starts

the 5 PG of 6 are UP now. we still have the last objet
fail and 1 PG fail

!!!


Comment by OVH - Monday, 03 October 2016, 17:59PM


2) we have lot of read/write on the ceph right now because
5 PG are UP now. the last one it's hard to make UP.


Comment by OVH - Monday, 03 October 2016, 18:00PM

UP!


Comment by OVH - Monday, 03 October 2016, 18:00PM

If your VPS is still down, please reboot it.