A bank’s data-centre model is increasingly the beating heart that sustains all of its services and allows it to respond to a changing market. As financial institutions move away from in-house models in search of data-centre optimisation, Jim Banks asks Ashley Davis, managing director of infrastructure services at UBS, how the bank’s changing priorities have led to new ways of thinking about managing its data needs.
At the core of all modern banking operations sits the data centre. Now, it seems that a bank is only as good as its data centre, hence the growing emphasis on optimisation and efficiency. Today, optimisation means many things, not least of which is enabling a bank to be agile and flexible.
Banks no longer see in-house or on-site data centres as the answer to their problems. A growing number of them are choosing co-location services, which place great importance on the choice of service provider, and more functionality is moving to the cloud, often in a hybrid model that blends private and public cloud capacity. Within such a model, finding the right recipe cannot be left to trial and error.
“‘Cloud’ means many different things to many different people. We have our own internal cloud, external cloud, as well as public and private. There are attributes of our computing environment that don’t need to be on our premises or computing fabric. So, on the basis of very careful selection, we do nominate some suitable workload types to go off-premises to a private cloud and we continue to evaluate cloud offering. Providing it meets our obligations in terms of security and is what we define as a compute core, then we consider that relevant for the cloud,” says UBS’s managing director of infrastructure services, Ashley Davis.
UBS is a complex organisation. As a global financial institution, it has four strands to its business: investment banking, wealth management, asset management and its corporate centre. Davis works in core engineering and is responsible for the bank’s global data-centre strategy and mission-critical buildings. It is his job to ensure that the bank’s infrastructure meets the needs of its primary stakeholder – the consumers of data-centre services, the network, the computing fabric, storage communities and, more importantly, application developers. UBS currently has 2,500 apps residing on more than 50,000 servers across its estate.
Given this complexity, UBS has made co-location a key part of its strategy. It’s running an active campaign to retire physical infrastructure in favour of a thin desktop community hosted on virtual back-end infrastructure. It already has more virtual than physical infrastructure as part of its plan to run its data centre without a sprawl of legacy systems across its remit.
“Our data-centre portfolio comprises owned-operated and co-location sites. The majority of our sites are co-location and that suits our requirements. Our contract terms can be between three and seven years, so we have the opportunity to flex and do life-cycle refreshes. That is our primary strategy. Owning and operating data centres comes with a lot of other overheads and, as we continue through our strategy, co-location would always be our first choice,” says Davis.
“In choosing the right partner, we are always looking towards a vendor’s covenant and we want a company that can offer some component of the technology stack, as well as flexibility and agility within that site. We are happy to cohabitate, we are happy to share, but we are still looking for segregation, and our own dependency on some of the mechanical and electrical estate,” he adds.
UBS has more than 20 data centres around the globe, as well as more than 100 sites deemed to be mission-critical. At the core of this network are six global hubs – two in Europe, two in the Americas and two across Asia. Across this hub-and-spoke model, the key priority is to ensure that the infrastructure is provisioned efficiently but, at the same time, can adapt to the bank’s changing needs.
“There are a number of IT challenges that we face. Around the data centre, a particular one is managing the envelope in terms of capacity. Many of our sites are co-located, though we have a few that we own and operate ourselves. The challenge is always around having sufficient capacity for major refreshes but, equally, not having an abundance of capacity that incurs cost to the firm. Legacy systems are also a challenge. We need to retire them and keep our environment fit for purpose. Agility is another concern. With technology life-cycle changes, we need to ensure that we can support emerging technologies. We need to manage that life cycle, which can consist of a mix of environments, from hardware that could have a ten-year life cycle to applications that could have a one-year-refresh life cycle,” Davis explains.
The new UBS headquarters in London are an embodiment of its strategy, which is built on the concept of ‘fat data centre, thin building’. This principle dispenses with on-site provision in favour of a more flexible and robust model.
“At our new headquarters, which comprises all our business units, the principle is ‘fat data centre, thin building’, so this building has no on-site data-centre facility. Looking around, we saw that there are a number of buildings in London where there is cohabitation of people and data. So, we set out from the beginning to separate people from data. There are no on-site facilities here for hosting any application servers and no SAN fabric. All we have onsite is our local network connectivity. The data centre was a critical path for us, particularly for this facility. All our back-end fabric is securely located off-site to meet our business-recovery objectives and satisfy the regulators that we have a robust recovery practice from an application perspective and a people perspective,” says Davis.
“This building satisfies our objective. It is also based on a thin-client model. No one has their own dedicated desk; they have an association with a neighbourhood. Therefore, no data is associated with an individual or a desk. All our data is hosted off-site inside the secure data centres. That eases mobility and increases our agility, certainly in terms of the mobility of our people across the building. From a recovery perspective, in the event of invocation, it is just a people-recovery game because all our data is now secure in our two UK sites. We would not be invoking a people and an application recovery. The concept has certainly met our original design objective, which was to ensure agility and support the mobility of staff. It meets our objectives in terms of maximising our office real estate and the efficiency of our data centres.”
Within the data centres, the demands on infrastructure have necessitated a different take on many technical aspects of the design in order to balance the key factors of capacity, efficiency and flexibility.
“In designing our HQ, we looked at what should come first – the building or the data centre, the network or the compute? The answer is that it is all of those things together. That was the intent – to get a unified design end to end for the building and our data centres. We look at everything, from what the next-generation network might look like to what the next-generation desktop will be. A thin client hosted in blades in the data centre, and all the other elements of the design, came together at the same time,” Davis remarks.
“The mission-critical design for our data centres was based upon scalability in terms of compute densification. We have real instances where we have racks running in the range of 14–16kW. The design of our data centres had to adapt. The life-cycle challenge I mentioned earlier is important here. We used to run racks at 3–5kW. We have moved to a vendor-based solution. We wheel in the rack, attach it to fibre and power, then the service is available. So, it is more of a modular design principle. We are running at levels where air is still optimal for us in terms of cooling. We are still using the design principles of a hot aisle/cold aisle layout, but it is far more modular in its design,” he adds.
Managing heat is crucial in any data centre, but in an estate that hosts an infrastructure with in excess of 50,000 servers, more than 70% of which are virtual, the heat burden could potentially be greater. Still, being able to cool that with air is essential for efficiency. There is certainly a need to move a lot of air in order to keep on top of cooling commitments, which brings challenges in terms of power consumption. UBS has not only moved to a different supply and return air temperature to ensure efficiency but has factored in the return on the other end of the ‘fat data centre, thin building’ model.
“Our dependency on power in our office buildings has gone down, as you would expect, because that workload is going into our data centres. We want to run these at the optimum level and, therefore, the higher utilisation means our power-use effectiveness is running at best-in-class levels. Downtime is another important consideration. ‘Always on and always available’ is the principle of fintech. Our data centres are designed to a 2N+1 solution to ensure maximum resiliency for all the supporting fabric, including mechanical and electrical,” Davis says.
2N+1 redundancy goes beyond the N+1 parallel redundancy often used to ensure that an uninterruptible power supply (UPS) system is always available. It doubles the number of UPS modules required to provide sufficient supply of power for essential connected systems, then adds one more.
“We design out downtime from the outset to ensure ‘five nines and above’ [99.999%] reliability,” Davis adds.
The in-depth approach UBS has taken for the design requirements of its data centres has also caused it to look at the cabling, which is the essential bedrock of any optimisation process.
“Copper is dead,” says Davis. “In the concept of the fat data centre, the cabling fabric has changed materially. The last generation had a high density of copper. Our model is highly fibre-based. IO densification is heavily fibre-dependent, which required a complete redesign of how we consumed cabling resources. We now have end-of-rack switches, fibre in-row and a fibre uplink to a core switch. Previously, there was a significant supply of patching and connectivity. It is now just about connectivity and fibre uplinks to our core switches.”
The approach UBS has taken has provided material advantages in its data centres and mission-critical buildings. It sets out a model of engagement with its vendor community that paves the way for other financial institutions to follow.