Routing Working Group
24 May 2023
9 a.m.
IGNAS BAGDONAS: Welcome to the start of the Working Group session, we have another day today and it's a short meeting and we really expect that we will flow in time, so therefore, we are just starting with the actual agenda.
JOB SNIJDERS: Thank you, welcome everyone. This is the Routing Working Group, we concern ourselves with all aspects of Internet routing and as we say here to us, payload is protocol overhead, the glue that keep together the Internet is BGP, not the DNS.
Oh, oh, well, let's discuss it afterwards. We have a tight agenda, so let me quickly introduce myself co‑chairs. We have Paul, Ignas and myself.
Well let's start with the first presentation, we will have a ten minute RPKI update. Thank you.
TIES DE KOCK: Okay, ten minutes, hope it works. Thank you, all. So I will try to figure out how this works. The next button is green. This is cool. So I have three main things I will discuss today, first of all RPKI adoption because something cool happened, second the developments of RPKI to RIPE NCC, slightly less cool, and roadmap. I am not sure if you guys heard, last Monday at 8:00 in the evening we passed the bar of having 20,000 certificates on the RPKI Trust Anchor which over the last few years and ‑‑ the green button doesn't work. Now it works. So 20,0004 certificates when I wrote this. We see a lot of growth of certificates over the years, both at RIPE NCC and other RIRs and not only more certificates but actually more space covered with VRPs in IPv4 and more important or more for the future, actually, IPv6.
But you can do IPI ‑‑ you can adopt RPKI and create certificates and objects but they don't really do anything without validation so that is more important than creating these objects but gathering data about RPKI validation is hard, you have four sets of data, you update a few of these projects where you talk to people about RPKI validation, how they implemented it, gathered that, but if you deploy ROV and one router doesn't get new VRPs, you may have kind of implemented RPKI but it doesn't actually work but if you do quantitative measurements, you take that case.
There's two main projects, gathering this information, you have APNIC's, gathering data from serves Whois to actual I balls and seeing whether it can be treated or not, academic method which in our eyes is the start of the art for R4, measurements to measure between two arbitrary end points and that way can cover many more paths.
On the maps on the next two slides, I will use data from APNIC Labs on RP filtering rates, you see here there's a lot of deployment of filtering in some countries, much less in others and there's a lot of effects at play there, there's data which need more granular data here to do this investigation because if you only test a few connections or a few users you may miss networks that have deployed it or not but you also see that adoption in countries really differs depending on whether big incumbents did employ ROV or not and this is our region but if you look at the whole world you have a similar picture.
Now, let's discuss what we have done over the last year, the main feature that we have deployed and getting used is publish in parent, we announce it at the RIPE meeting last year in May and we were able to deploy it in production in December and we see a lot using that and publishing objects there.
We implemented ASP A, because it's a draft you can create objects there, and these objects have actually ‑‑ in RPs and will enable us to be much quicker with deploying this when the standard is ratified. And we have also and I think you guys have all seen the updates, starting using status page, more of a cultural change in the whole organisation which communications team is leading but I really like we are getting small updates out to you all when we see something and inform you and when there's a bigger thing happening we will ‑‑ we do the same thing and afterwards we follow up with a little message like last week, I guess. I guess I can't really talk about us developing our RPKI stuff without talking about, also saying sometimes it goes wrong, we had a tiny glitch last week so I wanted to explain our development process a bit more. We try to be really careful with deploying and run stuff 24 hours before releasing it, we then have remote environment as closely as production environment and we have a lot of QAs things set up and work over the last year, run all major and few minor validators and we use API tests to make sure that a lot happening in these testing environment, we create ROAs with all strange configurations and CAs, the lead, delegated CAs remove them, create publication points, there's a lot of fun stuff there for the back end system and we do end‑to‑end testing.
We also do end‑to‑end testing for browsers so we just ‑‑ browser flows work and if you see/128 for your pop‑up that's one we use to measure the time between creating VRP on API and having it in production, in an RP.
So, what are we working on right now:
What we are working on right now, this is ongoing, is increasing our r‑Sync capacity, because r‑Sync is the fallback protocol for gathering RPKI objects but it's fall back and it means it's mostly idle, around 5400 connections an hour that mostly retrieve Trust Anchor certificates but when there's a disruption, we we need a lot of capacity when that happens and we are aware right now we do not have this capacity and we are revisiting this infrastructure to make sure we get that or we will get closer to that.
We are also working on improving the RPKI dashboard, we have started working on this and this will be a rewrite from the ground up because the current was started in 2013 using JavaScript and this is a rewrite, we aim for a better user experience there and that's one of the other reasons the software Engineering Department hired new engineers and desiren. We will first reimplement the dashboard with small improvements to get technical basis down and after this we will do a redesign.
Finally, we are working on compliance in the organisation and this is a project I'm not personally involved in so I want to get it first right, we have designed control framework, scope of which covers RPKI and supporting systems, we have completed gap analysis against control framework and we are now working on implementation.
Now things are ongoing will happen soon, I hope, probably most interesting bit which I have two minutes left, first of all we have received new online HSMs, we will replace with N shield connection 5C, the major improvement here for me is that we used to have one HSM in our staging environment and two in production and we will have two in our staging environment, why do you want to, it's a black box it should just work but in mid‑2022 we did software of HSM software staging environment for six months and in production we found it only works if you use low ‑‑ so your staging environment and production environments should be similar.
We are also asking objects, as I mentioned before we have implemented ASP A, we want to put that into our production API soon after our standard is ratified, we propose we have some actual use case that people have for RFC and I am not mentioning BGP Sec here on our roadmap but we no there's people talking about this and we are really curious about use cases would be unblocked by us you cannot do right now by having a delegated CA, we know about it, we have heard about it but lower on our roadmap so not on this block of work.
Finally, we will work on requested feature of integrated management of route objects in the RPKI dashboard so you can manage ROAs and route objects at the same time.
That's what we are working on and I have some time left for some questions, I hope.
JOB SNIJDERS: Perfect timing, thank you. Any questions?
PETER HESSLER: As you are familiar, the hosting of RPKI objects to the end users and ISPs is, has not been perfect in our ecosystem. RIPE, most of the issues with RIPE hosting has been resolved which we are very appreciative of but the other RIRs and especially the NIRs in other regions have had a lot of issues. Has there been any discussion about offering a mirroring service or a way of sharing these objects so we can fetch all this information from a single location?
TIES DE KOCK: I am aware of discussions ‑‑ I have been in discussions between RIRs where everybody is aware of the need of the high quality repository services, I think RIRs are doing very well. I have also seen issues with NIRs but I'm not aware of work on going objects for one place, it's interesting topic to look at I guess.
PETER HESSLER: Thank you.
JOB SNIJDERS: If I can add to that, I believe NIRs should use publishing parent, so like ID Nick or T W nick should publish into service and everybody will be better off.
AUDIENCE SPEAKER: Speaking for myself. I think a recurring theme that r‑Sync seems to be a bit of bottleneck for distribution, how big of a load is your serving seeing?
TIES DE KOCK: It'stive trivial except ‑‑ there was a discussion about how long clients should wait before falling back to r‑Sync, that's a random delay and that helped a lot but what we hit the, our storage peaked out at 200,000 per second that's what we could do and we just need to revisit it. I think it's possible to get the capacity there, but it's more hands‑on for us to do than hosting the IR P repositories, there we can use CDNs to offload the actual work.
ROB SNIJDERS: Thank you so much.
(Applause)
IGNAS BAGDONAS: Next on our agenda we have a talk about RIPE Academic Cooperation Initiative, something that allows for academia to present on their work that they are doing related to our industry and therefore I would like to invite Alexander Brundiers who will present his work on segment routing traffic engineering.
ALEXANDER BRUNDIERS: Hi everyone, as already said, it's my first time at RIPE meeting and I want to tell you a bit about my research regarding routing and traffic engineering.
I believe that most of you would already know what these two terms are but just to bring everyone to the same page, just traffic engineering in one condensed sentence, what we want to do here is control the path of traffic flows through our network in order to achieve various objectives. This can for example be to avoid faulty network elements, reduce energy consumption which is a rising theme in the traffic engineering community or to generally reduce the use of highly utilised links in the network. You can see for example if we are a failure on our network then some links might experience unexpectedly high traffic loads and then we might want to react with traffic engineering to relieve some of the load from these links. This can be done in various ways so one of the first I think was more or less metric tuning, adapting, MPLS and more recent tool is seg mount routing. And the general idea of segment routing is to ad‑interim destinations to a packet to control its path through the network, so normally we just have the IP address or just one final destination if you want and with segment routing we can add some kind of way points as in the example here, we just add like three labels to this packet here, U, O and S which means we first want to go to router U and O and to our final destination, S.
As you can imagine, the software is rather good traffic steering capabilities.
One terminology we need for my talk is the term of segment routing policy, which you can basically just understand as some form of rule that we configure on our routers that describes what kind of labels we add to what kind of packets. So, in this example here basically to ‑‑ at these packets to the packet ‑‑ to these labels to the packet, we would need to configure a policy on routing A that says something like if we have traffic from this node A destined to node S with add these labels to the packet.
And generally in ‑‑ from operator point of view we want as few policies in the network as possible, just to simplify the network management and make the network more clear, have less points of failure, basically. The problem however now with the current segment routing literature is that it just focuses on what we refer to as an end‑to‑end user of these policies so basically we have to configure these policies for individual traffic demands, on end‑to‑end fashion so from the start all the way up to the destination, and as you can see in this example here this is not that efficient because to deviate all four traffic flows here over the middle path we need to install four dedicated policies that more or less do the same across the network apart from these first hops, and this has the potential to increase the policy numbers drastically.
And the solution here, the solution here is the use of so‑called midpoint optimisation or IGP shortcut features with the basic idea of makes these policies usable by more than one demand. So the idea here is that in the example from before, we no longer have four policies but we have just one policy that we install on this node E here where we have some kind of, now we treat it as black box rule that something like okay all traffic that comes from the left side of this network and has to go to the destination marked here on the right, will be on to this single policy in our network. So, with this we basically have realised the same routing as ‑‑ it keeps switching forward ‑‑ I don't know ‑‑ I hope it stays there now. I can also explain it here, so the general idea of this is also a more realistic example is the usage innist P backbone networks, where we have a structure, multiple edge nodes that connect the network to other ISPs, customers and so on so to the outside world and these POPs or the traffic is mostly aggregated at a few core nodes that connect the network to the POP ‑‑ it jump forward again ‑‑ to the POP ‑‑ to the core network.
And if we want to traffic engineering here we have to configure very many policies, very many policies if we use these end‑to‑end policy functionality as you can see here this can get quite tedious and the idea with the use of these features is that instead we can just use them between these core routers here and can then control the whole traffic with very few policies between these core routers because generally for traffic engineering in these backbone networks the intra POP routing is not that interesting anyway and it's more relevant how we route traffic through the network core.
And the technical foundations for this already exist so we did not invent these ‑‑ this idea of mid‑product shortcut but there are similar ideas for MPLS I think and recently there are also implementation efforts from I think Cisco, Juniper and so on under various names that basically, is this idea, however they are not used yet, at least not to my knowledge so I am from academia so mostly I look into literature and there's not much record of the usage of this there. If any other ISPs or operators use it in their network internally I would love to hear about it in coffee break. But for now, to our best knowledge this is not used yet.
And yes, so we already have these technical foundations but we need algorithms that tell us how can we use this ‑‑
JOB SNIJDERS: The technical staff is working on it, it should be a seamless transition but our apologies for ‑‑
ALEXANDER BRUNDIERS: No problem, it tells me to hurry up, I think. We need algorithms to tell us how can we use this feature in the best possible ways and this is what I'm basically working on so what we are doing we get some topology data and some traffic data of these networks that we need into our algorithms and these algorithms then should say, should recommend us some kind of configuration that says something like okay we need to configure these policies to achieve a given traffic engineering goal like reducing the usage in the networks and this is done with some engineering and math that I won't cover here. I want to present some of our results.
What we focused on in our research were three central aspects, the first was the general solution quality so compared to the conventual end‑to‑end segment routing ‑‑ it just goes forward. We wanted to look at the general solution quality so do we lose any kind of optimisation quality when we use this features over current end‑to‑end segment routing? If we can really reduce the policy numbers with it and the last point is the general algorythmic complexity, how much time do we need to complete, useful in practice or more theoretical effort. For this we looked at backbone network or used from tier 1 backbone network and here we look at the max ‑‑ X Axis are different points of time in the network, and the Y axis shows the maximum link utilisation so most utilised link in the network, 1.0 would mean there's a link 100 percent utilised and we compare this to standard approaches to rate the performance and the first someone SpR so just the path as some kind of state‑of‑the‑art, I don't know if you can say that, but upper band that we want to improve upon and lower bound something called MCF, like the theoretically perfect routing that cannot always be achieved in practice but to assess the quality it can be used. And here we see that with our approach like the end‑to‑end segment routing, the Orange process. We are always able to achieve close to optimal solutions so we do not use that much of optimisation quality here. What is rather interesting now is the policy numbers we need to achieve these so with non‑end‑to‑end segment routing we require between 50 and 100 and in some outlier instances up to multiple thousands of these policies, while with our midpoint optimisation the same optimisation quality can achieved with just the single digit number or like 10 to 20 policies compared to this. We published this work last year and if you are interested in more results, see that paper.
And one last slide or last bit of information, the problem with the previously presented solution is that it takes quite some time so we need multiple hours to compute solutions which is acceptable if we want to do long term strategic optimisation on the basis of multiple days or weeks but when we want to react to failures this approach is not really suited because it takes so much time and to this we developed a second algorithm, more heuristic one, with the goal of achieving just good solutions within seconds and we have shown that with this algorithm we are able to remove congestion on our network more or less in sub‑second fashion in different isn't that so ewes like traffic changes, link failures and so on. Again this is also published at the conference this year, the conference will take place in Barcelona in two or three weeks so if you can't find it on‑line now, send me an e‑mail, I will send it to you in advance. Just the takeaway of this talk, basically what I want you to take away from here and this midpoint optimisation or features for segment routing are variation that are worth considering to use in your network because we are achieve similar optimisation quality as with conventional segment routing but we require substantially less configuration efforts to bring this out into our network and it also does not add that computation complexity so solutions can still be computed rather fast.
Currently I am also working on some kind of hybrid approach between these two so to combine them all into single approach but that's something for the future maybe for another RIPE meeting or so.
That's it from my side. Here you can find the publications, this work was partly supported by the Deutsch Telecom, thank you for attention, are there any questions?
IGNAS BAGDONAS: We have time for a couple of questions.
TOM STRICKX: Cloudflare, thanks for this presentation, super interesting. One kind of question for segment routing I guess there's two implements SR with MPLS and ‑‑ SR with IPV6 with do you take any consideration for label stack depth considering it's we are getting more MPLS capable devices but not all of them have the capabilities of five deep stacks, for example, is that something you take into account?
ALEXANDER BRUNDIERS: We took that into account. The results are shown here with a label stack depth of 2, basically one intermediate HOP and the final label of all these are done with two second routing.
TOM STRICKX: That's very impressive thank you.
PETER HESSLER: It's more of a comment but earlier in your slide you are describing what segment routing is and that was the most concise and clear explanation I have ever seen and answered several questions I have had so thank you very much for that.
ALEXANDER BRUNDIERS: Thank you.
AUDIENCE SPEAKER: There are two common ways to code your path with segment routing using /TPHO*D SIDs and adjacent SIDs, is your algorithm capable to calculate the optimal state if using ‑ SIDs
ALEXANDER BRUNDIERS: Currently we are not because sometimes it doesn't support EMCP splitting and so on and all this is done with node SIDs. Address SIDs are not used on our ‑‑ do not rely on them.
(Applause)
PAUL HOOGSTEDER: Next up we have got Job Snijders who will talk about archiving RPKI data.
JOB SNIJDERS: I want to talk about one of my projects called RPKI views, and I will explain what it does and with a request to the community for your support in one way or another.
RPKI views originated when I discovered there was a lack of common view on what the current state of certain RPKI aspects was. There would be discussions where different instances around the globe observed different output in the validated payloads and it was very Monty Pythonesque where somebody would say hey I think it's down and somebody else would say is, is not, is, and having our guy first to try and slurp in all RPKI data that is produced is a solution to have common data sets to reference and research when debugging problems.
It is moulded after the route views concept. I always liked with that, for decades they produced these snapshots of BGP data, as simple tower balls, very little interpretation and provide that to the research community but also the operational community, has used this data, do with it as you like. So how RPKI views differs? It doesn't connect BGP data, it only collects RPKI data. What it captures is the raw DR encoded ‑‑ it also contains interpreted output in both JSON and CBS format and open metrics file detailing the synchronisation process towards different publication points. As metadata there's a log file that shows what the execution run looks like at that point in time, a public key to verify the integrity of the tower balls and directory listings and there's this for purposes of ensuring that you have ‑‑ the arc rife as complete as it can be in its intended state so the public key is more for check purposes than authentication.
Some feedback I have received so far on RPKI views is, there was one instance where there was an outage for a customer and the customer said we did not change anything and through RPKI views we could conclude that the customer had in fact created a ROA but with the wrong original ASN and this hampered their ability to propagate the route in default free zone and having an independent archive to point you created a ROA, you cannot deny this, is beneficial, and then recently a number of researchers used the data to analyse the delays in publication times, super cool use case and I'm excited to see what other use cases and things happen in the future with the RPKI views data.
So there's a few ways to access the archive, one is HTTP, you can go to the website, you will see pointers to the individual nodes and if you click through it's organised by year, by month, by day and then you see the tower balls each time stamp according to when it was finished.
You can see also access all the data through r‑Sync, each of the RPKI views nodes not only offers HTTP but r‑Sync daemon instance and this is especially for researchers is much nicer because you can set up a job and just keep syncronising and my hope is that people will keep copying this data into places so that future generations can take a look at this.
The contents of such TarBal are as follows, there's basically inside each TarBAl two directories, one is called data and that contains the DR encoded signed objects and certificates and CRLs and the other is output and output has the validated data and log files and metric files.
Why this mechanism is useful, the RPKI is a distributed database that might look different from different places on the planet depending on ‑‑ depending on the exact moment you would try to syncronise and there's a lot of factors that come into play. So collecting RPKI data from different places on the planet offers us more insight into what exactly happens during incidents or can guide us into understanding how future extensions to the protocols should work.
There's a few notes for researchers. Each of the RPKI views nodes runs on its own schedule, some of them run every 30 minutes because of resource constraints, some of them run every ten minutes, and researchers would do well to collect tar balls from ‑‑ this is exciting presentation system ‑‑ researchers would do well to syncronise the tar balls from the different nodes around the planet and compare the results with each other, it's super useful to not focus on one single location but look at all locations and try to create a narrative from that. Currently, the RPKI views nodes are located in Amsterdam, Kansas city, Tokyo and Zurich and I would love if there are more nodes in different continents on this planet.
Each snapshot, each TarBal contains only valid data so it makes no attempt to store data that maybe at some point in time was valid or perhaps never was valid, each node is a validator and it slurps in the data, applies to cryptographic verification processes and the TarBal is essentially a copy of the local cache of the RPKI validator. This is ‑‑ I have ‑‑ I set it up this way because if I store everything that's not valid there's a bit of the risk that the archives might grow beyond what I can handle, and I also think it's simpler for everyone to understand what is happening if we make no attempt to archive invalid data.
And if you are a researcher and you are going to do your thesis on RPKI and you are interested in using RPKI views data it is perfectly okay to ask me how exactly I collected the data, what the storage process is or what certain quirks or discrepancies in the data might mean, I am super happy to help you understand the archive better and make got a use of it, so, if you are going to use this data, talk to me, please.
Currently, RPKI views is supported by Internet Initiative Japan, my own resources, August Internet, A2 B Internet and digital ocean and it is really cool that everyone came together and provided resources, either CPU resources or storage resources or bandwidth to make this possible.
And my request to the community is, if you have spare resources laying around or if you are a big Cloud provider like AWS or, you know, other big ones, please talk to me. I can really use CPU resources and storage resources, the current archive already is 50 terabytes and it is growing and I cannot keep storing the data indefinitely without additional help so if no additional resources are made available, it would simply mean that other data starts to get deleted and we lose a little bit of history.
Now, with that, I will open up the microphone for questions, comments, remarks, concerns and I look forward...
AUDIENCE SPEAKER: Have you made any attempt at deduplicating the data, have you tried on a system or something like that?
JOB SNIJDERS: I appreciate the suggestion to deduplicate the data, and it is somewhere on the to‑do list to maybe optimise in that direction, I hope to at some point in time build a database where all the files are indexed by their ‑‑ 252 hash and maybe generate archives on the fly but the advantage of the system simplicity for researchers and operators trumps the idea of deduplication system because right now each is a perfectly contained snapshot in time without dependencies on anything else in the system, so there is opportunity for deduplication, definitely, but I also ‑‑ there's trade‑offs, so like one of the nodes is actually a Linux box that uses ZFS as it's underlying system and that takes the ZFS snapshot every time it executes and then produces a TarBal based on that, so it differs a bit from node to to node how exactly data is stored in the back‑end but it's an interesting quest, how to store this data in the long‑term, in the long run and keep it manageable.
AUDIENCE SPEAKER: Tim, NLnet Labs. I have a question about the invalidator that's rejected. My concern is that if people use this as a public archive of what is there, then it may have some bias, for example somebody may think they published a ROA and then it may not show up on /TH‑PB archive and then you get discussions. So, I think it depends also on how well you log what happens but I think it could also help to have invalid objects in there but flagged as invalid so people know where to look.
JOB SNIJDERS: You make an interesting suggestion and I will use this opportunity to give a brief sneak preview.
So here I said hey, you can use r‑Sync to ‑‑ I will remove your batteries ‑‑ can you control the slides and go to the one I intended, a little bit back so here I show you can use r‑Sync to syncronise the full thing and if you use this end point you will slurp in 25 terabytes of data. The first two lines, 2022, 2023 is what's archived from the perspective of this Japanese node. Amber dot SARS .net is a copy of this node in Zurich and there's comparison views which archives RPKI client and Routinator and produces a diff between the two every iteration to overcome the issue of hey, is it a problem in one specific validator or a multiple? And there is the RD DP, r‑Sync collation end points, one in Australia and one in Amsterdam, where I launch instances that concurrently that do RRDP only, RRDP with fall back to r‑Sync and r‑Sync only and there's a three‑way comparison what they observed at a given moment in time, but these are relatively new data sets I am collecting so I'm not ‑‑ they are my ‑‑ there might still be work on the exact format or how I approach it but I do try to provide a dataset that offers geographical diversity, validator diversity, sync nation diversity and do so in a consistent manner to allow longitudinal ‑‑ long term studies of the RPKI ecosystem.
PAUL HOOGSTEDER: Thank you, we are running out of time. We have to close the mics, sorry. We get on with our next speaker, who will be Ben, I think.
(Applause)
He will given an update on BGP tools, he has presented this a year ago in Berlin and now that's his story on how he spent the year working on that.
BEN CARTRIGHT‑COX: Thank you, this is this about the realtime collector on BGP tools one year on. Last year in Berlin I presented BGP tools as a whole but also I mentioned oh cool ‑‑ so, okay. So, last year I talked about not only ‑‑
JOB SNIJDERS: Maybe we should switch to asking them to go to the next slide.
BEN CARTRIGHT‑COX: Next slide, please. So, last year I talked about moving to realtime collecting so I was moving away from RIPERIS and reviews as data sources and decide to go and build my own.
So if you haven't seen it already, this is BGP tools, it's just effectively an easy way to go and look up stuff in realtime, you can type in ASN, prefix, a whole bunch of of stuff and the site gives you nice easy to use views of the Internet and hopefully in realtime using its realtime route collectors.
So BGP tools also has had a few improvements since then so one of the nice new things is global looking glass so you can use it much more similar to things like reviews, the interfaces, there's also web interface if you don't have a client on hand.
So, there is also much more work has been put into internet exchanges points which I will be getting on to later and so internet exchange not only show Mac address data which gives you the top vendors which is really interesting, for example certain regions much prefer micro tick, certain more in the European regions we prefer Cisco and Juniper which I think is fascinating.
In addition, there are much bigger improvements to the internal size looking glasses if a ‑‑ you can now go to that ASN's page and use if the AS ‑‑ if the network has opted into it, you can use BGP tools as their looking glass as sort of easier way to go and find various networks's looking glass. Link in an account and RIPE Atlas API key there's a nice convenient, if you are looking to do an MTR from single ASN you can get an MTR interface with one click.
Finally there's no good motivator than a good competition, the BGP tools also does ranking. Now I must point out only one of these rankings can be changed by feeding me data but the rest, you have peer count which is the one if you feed me data then BGP tools will show you what it can see and the rest is like AS cone which is what CAIDA uses, I can't remember the name, the second ‑‑ last two are eyeball population which are answers the question if you are going into a new region what are the biggest eyeball ISPs, these are estimates based on ad traffic and there are domain records which could answer the question of what are the biggest content provides in a region.
I'm not going to read these all out the site does community values, I am currently experimenting with client side software so if you don't want to run things through the websites you can provide people just a real do Monday to run inside your network to do trace routes and things like that, (daemon) the rest can be read through on the slides later on.
Core points. If you saw the ‑‑ remember from the slide before, then there were 84 sessions established when I presented at RIPE last year or last year, yeah, now I have 940 and the site is still practically realtime. However I do feel like I am starting to reach the limit of networks that will be ‑‑ which are willing to eBGP multi hop over the Internet which is very understandable, for a whole bunch of reasons ISPs have ACLs that do not ‑‑ are not compatible traffic out of their edge. I also am starting to believe there is a chronic lack of data visibility for ISP route servers, they have a huge sway on routing decisions for networks and often are imported unfiltered, I think there should be a lot better eyes and I struggle to find good data from them, BGP tools is now moving into internet exchanges.
So peering then first comparing the strategies, so RIS and route views live on IXPs the people feed session, a lot of people were feeding their customer cone because they haven't fully communicated what the purpose of the route collector is or just over time some engineers, we are accidentally sending this entity a full table, we should do that because we are giving them free transit, so IXP collectors are a different challenge to making sure that you are keeping on eye on all of your feeds to make sure they remain useful and over time, a lot of them being co‑located on the internet exchanges, hardware researchers are difficult, some of them at capacity limits.
Rough comparison. So RIPE currently has around 1,500 BGP sessions online and about 372 v4, v6 tables, 407, according to their own calculations, that was made a couple of months ago so might have changed by a little tight. Some of these sessions have issues but a lot of these are genuinely really useful views of the Internet and because RIPE is in regions, RIPE and like South Africa which have chronically low visibility on route collection projects these are useful pieces of data.
So, BGP tools is currently 99% BGP multi hop only, I have 940 sessions online and 634 v4 tables ‑‑ 920 v6 tables, this is possible mostly because multi protocol is supported on BGP tools so you can feed v4 and v6 in the same TCP connection, allows you to feed all your routers tables at once which does help the visibility of various transit providers.
So, one of the other problems with IXP route collection there is a massive bias to Hurricane Electric. They are on almost all of the large IXPs, probably all of them, and they provide you around 180,000 peered v4 routes that you are likely going to import and then use, the problem with this is if you then feed it right back to the route collectors because if you are on the exchange you are probably going to feed back that table straight back to a collector so when Hurricane Electric gives you something of peering they have masked away a whole bunch of interesting transit routes which is a shame but obviously not their fault, it's a victim of their success, to some degree.
The other problem with IXP route collection it's really expensive if you don't have relationships with the IXPs, a membership fee itself can be more than the entire operational cost of running a collector and that's not even excluding things like the cross‑connect fee, in some regions that can be extremely expensive and the growing cost of power is increasing concern, so it's really interesting going through internet exchange provider on peering dot expose which ranks to some degree pricing on internet exchanges and price per megabit depending on your usage, the problem with using things is that BGP tools will pick up a 1 gig port and use ‑‑ 1 megabit all it's exchanging is BGP traffic so a lot of this is not economical if there isn't a special deal with internet exchanges.
So, what is the smallest, cheapest, most insane thing you can ship to a willing IXP that could solve some of these problems.
So this is a smart, made by a Russian company which is very interesting.
So you don't need a cross‑connect, this goes into the IXP switch, the switch is the power supply so you don't need to pay for power, I guess you are hitching a ride. It's technically no worse than a standard SFP and there are about 150 US dollars, run a full version of Linux, about the same computational power as original raspberry PI so single, it's also a bit nuts, people when I explain this to them are a little apprehensive which is very understandable, there is no way you are plugging that into my internet exchange, but there /SR‑BG some takers so fair game to them. ‑‑ most have and so getting a one‑off to replace an optic to all these magic pixie object particulars is not particularly great.
The other problem I forgot to mention is that the optics are made by a company who was Russian and for sanctions reasons are a little apprehensive at buying those now, there are other solutions so require you to get more creative, the previous was by design, there are other computers that aren't advertised as computers in SFPs, this is a with Mac optic you don't need an ONT in your home, you can just stick your G‑PON line into one of these object particulars, it's doing all of the logic to convert those to synchronize up with the optical line system or whatever so really cool stuff. These are running Linux in the in the back‑end, inside of these are full functioning Linux system.
These are obviously a lot more, they don't need to do that much thinking, about 400 megahertz, it's pretty basic. Makes this a really interesting and full challenge to programme for. Thankfully rather than programming MC which I am not good at, modern languages like ZIG has mostly functional target, write reasonably safe code in minutes without using C. To use this as a general Linux box you have to make sure that G‑PON optic doesn't fire so your switch won't bring up the port so the vendor for these which I think is Leo Labs in Poland has been really keen and helpful so a masstive shout out to them. I think they are just amused at what I'm doing. Similar technology to this is also available from knock co‑and others and, there's two competing design, hey way and and Nokia have a similar design, have the same credentials, so use the same user names and passwords, they are cheaper.
So the actual preference, BGP tools has, some IXPs have virtual machine infrastructure, this is by far the easiest method of deployment, BGP tools are run internal relay for very low CP E requirements, you can hitch a ride off someone on the internet exchange. Those magic optics are very easy and tiny, you can ship them easily, if one disappears into the post or customs, so obviously they are minorly scary, a lot of people do not want to put those into their switches, there's only a 1 gig version of those, because it doesn't particularly make sense any more when you can buy 1 gig broadband service. At worst I will happily ship out and pay for power if necessary, I want to hit many internet exchanges as possible to conserve power costs.
You may have noticed on these optics it's not really possible to store modern Internet table so instead of storing ‑‑ one of the differences between rip RIS and reviews is BGP tools back halls all of its BGP session data back to London where it's hosted, this is because the way BGP tools is designed in order to give you the instant feedback all the BGP route collectors and all the data has to be roughly about 3 milliseconds within the web server to enjoy a normal experience.
So the current progress is I brought NL IX up and thanks to the JINX, DINX, Johannesburg, inner own toe ‑‑ there is a whole bunch more, I had great conversations with people here to bring up agreements. In addition, separate root server feeds so minute app, in Kansas have come up with just route server feeds so the difference between these two is the top point, you can peer with BGP tools on these exchanges, the bottom one I am only getting root server feeds.
If you haven't already set up feeds I would encourage you to do so, you can login through PeeringDB if necessary or sign up for an equal account, you can instantly set up multi hope sessions, you can't set up full sessions yet.
Any questions?
IGNAS BAGDONAS: We have severely running out of time so close the mic. Please on to the mailing list.
(Applause)
JOB SNIJDERS: It turns out this clicker and the clicker in the other room are operating on the same frequency or signal, so there's been a lot of synergy between this session and the other session.
I am stepping down as Routing Working Group co‑chair and I believe Ignas ‑‑
IGNAS BAGDONAS: Other fellow co‑chairs for the reason you are stepping down, we would like to hand a small parting gift for your service to the community and then you can carry on with selecting another co‑chair. Thank you.
(Applause)
JOB SNIJDERS: Thank you so much. Selecting a new co‑chair, super exciting and I'm very happy to report that multiple capable individuals stepped forward to make their time and energy available to this Working Group. And given that we have multiple candidates we need to somehow pick which one is going to be the next new co‑chair. So we devised a procedure that hopefully is fair, adequate, unambiguous at some point in the process. Each of the candidates is given one minute to introduce themselves and while they talk there will be one slide behind them up on the screen to support their introduction.
Once all candidates have introduced themselves, we are going to ask the room three times to do IETF‑style humming in favour of a candidate. So if you prefer one candidate over the other, hum after that candidate's name is called out or stay silent if you prefer different candidate. And you can of course hum multiple times if you like all candidates.
Then the ‑‑ myself, Ignas, Paul, Mirjam, will then deliberate to see if our understanding of the loudest of the humming is unanimous or not, maybe at this point of the process there's clear support for one candidate over the other candidates, if there isn't we will proceed with a random selection process because at that point in time it means that we have multiple qualified candidates, multiple candidates supported by this Working Group, which means there is no wrong choice. Mirjam, you want to add something.
MIRJAM KUEHNE: I have a question about the first part. You said you can also hum multiple times if you like all candidates. You think that is a good step?
JOB SNIJDERS: I think I cannot stop them.
MIRJAM KUEHNE: But you don't have to take it into the process. I would suggest, what I would suggest you hum after the name that you would prefer as the next Chair because you are only looking for one person. Thank you.
AUDIENCE SPEAKER: Can we maybe do a test hum for two pretend candidates so people understand the process,
JOB SNIJDERS: Warren will be our our test subject.
AUDIENCE SPEAKER: Between Warren and Mickey Mouse.
JOB SNIJDERS: If you are in support of Warren being co‑chair, please hum. All right, not too bad. So yeah, the idea behind humming is that it's not voting, we are not looking for absolute numbers but it's to gauge the temperature in the room in a semi‑anonymous way. We are going to proceed to the introductions of the candidates, each of them will have ‑‑ one minute. Tim, where are you? I will start waving when a minute is up.
TIM BRUIJNZEELS: Hi, why me? Let me so I have been working in this community for quite some time, first as a RIPE NCC employee but now five years in NLnet Labs. I have done a lot of work in the IETF, mainly on RPKI. I would very much on a personal level like the opportunity to experience the community from the other side as a Chair. I also think it's a very important job to essentially facilitate the community to speak their mind. My own focus is on RPKI and routing security, I think that that knowledge and my exposure in the IETF may add, may complement the other Chairs. On the other hand, my hands‑on routing experience is limited so if that's what you are looking for then I'm not that candidate.
JOB SNIJDERS: Next slide please.
ALESSANDRO IMPROTA: Hello, I am engineering manager at Catchpoint since 2019, I was ‑‑ I was research shall all my life before I joined Catchpoint so I focus on routing, BGP data, RPKI data, things like that, and during the years I folks mostly on collecting data and approached with my colleague Luca, up to 2019.
So we started from scratch with that.
This is my way to try to contribute back to the community, after taking a lot doing the last years but I have been around since RIPE 67 and I took a lot of knowledge from here, so I hope this is going to be appreciated. Thanks.
(Applause)
JOB SNIJDERS: Ben Cartwright Cox, would you please come to the stage.
BEN CARTRIGHT‑COX: Hi, as you probably have already heard I'm Ben. You may remember me from a few years ago with NLNOG talks about RPKI stuck routes, you may know me from the person who played battleships with BGP, I am sorry with that. I am obviously as you saw currently working and maintaining BGP tools, I am very interested in routing stability and security, mostly for my own sake. I previously worked at Cloudflare and have worked on the other side of the tech coin working at UK bank as revitalizing their data centre infrastructure, and to give back to the RIPE community who has helped me a whole bunch in the past. Thank you very much.
(Applause)
JOB SNIJDERS: There now is a step in the process where we apply a highly scientific procedure to interpret the room's preference for one candidate or another. Let's start with first to last, if you would like Tim Bruijnzeels to be could chair of the Routing Working Group please hum.
(Humming)
If you would like Alessandro to be co‑chair of the Working Group, please hum.
(Humming)
If you would like Ben Cox to be co‑chair of the routing Routing Working Group, please hum.
(Humming)
I would like to invite my co‑chairs to the front of the stage and Mirjam so we can have a short discussion.
MIRJAM KUEHNE: I think ‑‑ it was a difficult choice, all of the candidates got support which is great to see. Thanks for all the candidates to step forward, I think this is great to see from the community. Number 2 what we clearly heard was the lowest, so Alessandro.
It was close between 1 and 3, so between Tim and Ben. But we all agree that we heard a little higher hum for number 3, which is Ben. Thank you, thank you, Tim, for volunteering and thank you Ben and welcome as the new co‑chair of the Working Group and thank you, Job, for your good work.
(Applause)
JOB SNIJDERS: This concludes the Routing Working Group session at RIPE 86, see you at RIPE ‑‑
IGNAS BAGDONAS: Not really, the real conclusion will be just in a moment but we have another important topic and the milestone of the Routing Working Group, it's 20,000 off ‑‑ achieved just recently?
JOB SNIJDERS: Intermediate CA certificates?
IGNAS BAGDONAS: Yes. Therefore we have a cake to celebrate that occasion. So, please everyone in the room come here enjoy this moment.
JOB SNIJDERS: Thank you all,
(Applause).
