Vault login failures for some users

Incident

August 22 2024,5:46am PDT

Vault login failures for some users

Status: closed

Start: August 15 2024,6:10pm PDT

End: August 15 2024,7:45pm PDT

Duration: 1 hour 35 minutes

Affected Components:

Vault CRM Veeva CDMS Veeva Vault Veeva Align Vault CRM Align MyVeeva (ePRO, eConsent) Veeva SiteVault ePRO Vault MyVeeva-US VA-US MyVeeva-EU Vault-US PODs SiteVault-US VCRM-US PROD VCRMA-US ePRO Vault-US CDMS-US VA-US-2 MyVeeva-AP Vault-EU PODs SiteVault-EU VCRMA-EU ePRO Vault-EU CDMS-EU VCRM-US SBX VA-US-3 Vault-AP PODs SiteVault-AP VCRM-EU PROD ePRO Vault-AP CDMS-AP VA-US-SBX VCRM-EU SBX VA-EU VA-EU-2 VCRM-AP PROD VCRM-AP SBX VA-EU-3 VA-EU-4 VA-EU-5 VA-EU-6 VA-EU-SBX VA-AP VA-AP-SBX VV1-4001 VV2-5001 VV3-6001 VV2-10 VV3-33 cdb-1 cdb-101 VV1-1082 VV3-3122 VV2-2097 VV2-5000 VV1-4000 VV3-6000 VV1-1187 VV2-2145 cdb-201 VV1-1179 VV2-2144 VV3-3130 VV2-2149 VV1-1 VV2-27 VV3-34 cdb-3 VV1-1110 VV2-2129 VV2-2092 VV3-3121 VV1-1189 VV1-1182 VV1-2 VV2-28 VV3-3048 cdb-5 VV1-1111 VV2-2142 VV1-1191 VV1-1188 VV1-3 VV2-29 VV3-3057 VV1-1160 cdb-7 VV1-4 VV2-30 VV3-3064 cdb-9 VV1-5 VV2-31 VV3-3086 cdb-11 VV1-6 VV2-32 VV3-3096 cdb-13 VV1-7 VV2-35 VV3-3099 cdb-15 VV1-8 VV2-41 VV3-3120 cdb-302 VV1-9 VV2-44 cdb-1002 VV3-3121 VV1-11 VV2-2047 VV3-3122 cdb-1004 VV1-12 VV2-2050 VV3-3123 VV1-17 VV1-13 VV2-2056 VV3-3124 VV1-22 VV1-14 VV2-2060 VV3-3125 VV1-37 VV1-15 VV2-2063 VV3-3127 VV1-1090 VV1-16 VV2-2070 VV3-3128 VV1-1127 VV1-18 VV2-2072 VV3-3129 VV1-1134 VV1-19 VV2-2075 VV1-1139 VV3-3130 VV1-20 VV2-2080 VV1-1148 VV3-3131 VV1-21 VV2-2083 VV1-1158 VV1-23 VV2-2085 VV1-1159 VV1-24 VV2-2087 VV1-1161 VV1-25 VV2-2091 VV1-1174 VV1-26 VV1-1175 VV2-2092 VV1-38 VV2-2095 VV1-1193 VV1-39 VV2-2097 VV1-1194 VV1-40 VV2-2098 VV1-42 VV2-2120 VV1-43 VV2-2121 VV1-1045 VV2-2122 VV1-1046 VV2-2124 VV1-1049 VV2-2125 VV1-1051 VV2-2126 VV1-1052 VV2-2127 VV1-1053 VV2-2128 VV1-1054 VV2-2129 VV1-1055 VV2-2130 VV1-1058 VV2-2131 VV1-1061 VV2-2132 VV1-1062 VV2-2133 VV1-1065 VV2-2134 VV1-1066 VV2-2135 VV1-1067 VV2-2136 VV1-1068 VV2-2137 VV1-1069 VV2-2138 VV1-1073 VV2-2139 VV1-1074 VV2-2140 VV1-1076 VV2-2141 VV1-1077 VV2-2142 VV1-1078 VV2-2143 VV1-1079 VV2-2144 VV1-1081 VV2-2145 VV1-1082 VV2-2146 VV1-1084 VV2-2147 VV1-1088 VV2-2148 VV1-1089 VV1-1094 VV1-1110 VV1-1111 VV1-1120 VV1-1121 VV1-1122 VV1-1124 VV1-1126 VV1-1128 VV1-1129 VV1-1130 VV1-1131 VV1-1132 VV1-1133 VV1-1135 VV1-1136 VV1-1137 VV1-1138 VV1-1140 VV1-1142 VV1-1144 VV1-1145 VV1-1146 VV1-1149 VV1-1150 VV1-1151 VV1-1152 VV1-1153 VV1-1154 VV1-1155 VV1-1156 VV1-1157 VV1-1160 VV1-1163 VV1-1164 VV1-1165 VV1-1166 VV1-1167 VV1-1168 VV1-1169 VV1-1170 VV1-1171 VV1-1172 VV1-1173 VV1-1176 VV1-1178 VV1-1179 VV1-1180 VV1-1181 VV1-1182 VV1-1183 VV1-1184 VV1-1185 VV1-1186 VV1-1188 VV1-1190 VV1-1192 VV1-1193 VV1-1194 VV1-1195 VV1-1196

Update

August 15 2024,6:10pm PDT

Veeva Vault PODs in Region Veeva Vault are currently experiencing degraded performance for some SSO customers. Veeva engineering teams are working to return the service to normal as quickly as possible.

Update

August 15 2024,6:58pm PDT

The latest hotfix is the cause of this issue. The team is going to generate an emergency deployment to QA and then it will executed in production.

Resolved

August 15 2024,7:45pm PDT

Affected Veeva Vault PODs are now fully available with all services operating normally.

Incident Summary

August 16 2024,8:06am PDT

An update regarding this incident.

A change applied to Vault Auth on Thursday 15 August 2024 at 5pm PT led to certain users being unable to access Vault.

The issue was not POD-specific.

The issue arose from a defect that incorrectly enforced case sensitivity when a Federated ID user name is verified against the Identity Provider. Previously the check was case insensitive.

Impacted users were those with Federated ID user names that do not exactly match the IDP value.

A Root Cause Analysis is being prepared.

Incident Summary

August 16 2024,8:23am PDT

To better reflect the incident scope and impact, the title of this Trust entry is changed from:

Vault Performance Degradation affecting all PODs in Region Veeva Vault

to:

Vault login failures for some users

Incident Summary

August 22 2024,5:46am PDT

A brief note that the Veeva Quality & Compliance team has adjudicated this topic as requiring an Incident Report, which builds upon and supersedes an RCA. The RCA content already well under way will be augmented by a deliberative process leading to formal CAPAs tracked by Veeva Q&C. This extends the timeframe of a detailed report by several days owing to the heightened review process and the development and approvals of formal CAPAs as required by an Incident Report.

The Incident Report is a top priority. We will update this Trust entry within several days with the status of the Incident Report.