Affirmation of Commitments Consultation Wednesday, 27 October 2009 ICANN Meeting Seoul, Korea >>PETER DENGATE THRUSH: Ladies and gentlemen, could you take your seats, please? We're going to get going in just a minute. If you're here for the session, could you close the door, please. Thank you. Just while we're waiting to begin, just note, there is a full bank of interpretation going on. You'll be able to speak in any of the languages that we're using. If you need translation interpretation, make sure you have one of the translation devices. Right, let's begin. Ladies and gentlemen, good afternoon. Thank you for coming to this short session on the Affirmation of Commitments. Because of the time, I'm going to assume that most people know the history of the green paper, the white paper, the memorandum of understanding, the signing of the Joint Project Agreement, and its ending. And we're moving forward into the new phase that the Affirmation of Commitments provides. It provides for a major maturation, a growing up, of ICANN. But it is going to require all of us to make it work. It's going to affect almost every aspect of ICANN, because the reviews that will come toward the end of this short presentation go right across the organization and look at how it performs. We've had a series of bylaw reviews which go this way through the organs of ICANN. We look at the GNSO, and the board and the NomCom. Now these reviews look horizontally across at functions and performance areas. What's good about the Affirmation of Commitments, from my perspective, is that it maintains and continues the kind of obligations that I've called the generic obligations that we've always had. So we continue to make the promise, for example, that our decisions related to the global technical coordination of the DNS are made in the public interest and are accountable and transparent. It's an absolutely primary obligation there. Made in the public interest, and are accountable and transparent. And, of course, our usual mantra is maintained. We promise to preserve the security, stability, and resiliency of the DNS. We're also going to carry on promoting competition, consumer trust, and consumer choice. And we're making sure that the international -- there is international participation in the DNS. And then there's -- it starts to tighten up from those very high- level generic obligations to some that relate to our processes. And we have to use transparent and accountable budgeting processes. We have to go to fact-based -- or continue, rather, using fact-based policy development. We have to continue the cross-community deliberations. And we have to publish studies of the impact on consumers and others of our decisions. We have to use responsive consultation procedures. Well, anyone who's been through the new gTLD program can see how well I think we understand that and we're doing that. I just have to stop and praise the staff for a moment for the way that the public comment has been handled in the new gTLD process. From an area that ICANN used to get considerable criticism, I think we're now doing this at a superior level. And we have to continue publishing an annual report. The mission, of course, is still the same. We're going to coordinate the Internet DNS. And by that, we define that as being the names, the numbers, and the protocols. We're going to remain a not-for-profit corporation headquartered in the United States, we're going to meet the needs of the global Internet community, and we're going to operate as a multistakeholder private sector-led organization with input from the public, for whose benefit ICANN shall act. So same kind of things that we've had. But the key to this is, the way we are going to report and check on how we are meeting those obligations. Instead of being done by way of the United States government calling for reviews, it's going to be done by ICANN conducting reviews. There's four of them, transparency and accountability; preserving security, stability, and resiliency; promoting competition, consumer choice, and consumer trust; and in relation to WHOIS. At this point, I'm going to hand over to Marco, who is our professional expert in conducting reviews. Marco works closely with the SIC. And you will be familiar with his work in relation to the restructuring particularly going on of the GNSO. Marco, tell us about the affirmation reviews. >>MARCO LORENZONI: Thank you very much, Peter. First of all, to the requirements contained in the Affirmation of Commitments about reviews. We are committed to four different reviews: Accountability, transparency; security, stability, and resiliency; competition, consumer trust, and consumer choice; and WHOIS. Each of these reviews is broken down within the text of the Affirmation of Commitments in different areas of analysis, six areas of analysis under the first area of review, and three each for the other three reviews. And those areas of analysis include aspects that are correlated, but in some cases, even distant one each other and complementary. For instance, under accountability and transparency, we have to give an answer about the board governance structures, about the role and effectiveness of the GAC, about how ICANN receives public input, about how the decisions taken by ICANN are supported and embraced by the Internet community and by our respective community of reference, about the policy development process, and about the ability of ICANN as a structure to implement the recommendations that were coming from the previous cycle of review. Much more homogeneous, the area of reviews under the "Other Reviews," for instance, under "Security, Stability, and Resiliency," we will have to answer two questions, such as security, stability, and resiliency of the DNS; about adequacy of the contingency planning; and about the clarity of the processes that have been drawn for this aspect. On the third review, the so-called three "Cs" -- competition, consumer trust and consumer choice -- we have to assess what is going to be the effect of the introduction of the new gTLD under those three parameters, effectiveness of the process, and effectiveness of safeguards. Finally, WHOIS. There are three questions: Policy effectiveness, meeting needs of law enforcement, and effect on promotion of consumer trust. Not all of those reviews have to start, fortunately, at the same moment. But the time line is very tight. This Gantt shows you the time line of the first two cycles of the Affirmation of Commitments and of the ongoing activity of organizational reviews that is going close to the end of the first cycle but will continue in the future with the assessing of the effects of the changes. If we envisage some preparatory actions from now till February/March of the next year, we have a first part of the activity, which is ensuring accountability and transparency of the organization, that needs to be delivered by the end of 2010, while contemporarily starting the review on preserving security, stability, and resiliency, due to start in October 2010. The same for WHOIS. Still uncertain the date of starting of the review on promoting competition, consumer trust, because it will depend on the date what the new gTLD program will be adopted. So it will be one year from the beginning of the activity. Any kind of expert in project management will see from this chart that this will pose very serious problems in assigning of the resources. So we will have to work on the time plan in order to have not too many peaks of activities collocated in a single moment. As I was saying, timing is already urgent. If, from the previous slide, we concentrate only on the preparatory activities, we need at least to start a few activities immediately, which are the preplanning, the drafting of the processes to carry out the reviews in order to go to public consultation and to receive as soon as possible inputs from the community on these processes. And this can be done, for instance, in January, late December or January, and to pass through the following implementation actions, which are hiring of external consultants to support us in the carrying out of the review, which is setting up the teams of volunteer members. This is a very important point that is made by the Affirmation of Commitments. As you know, the review teams will be made up of volunteer members of the community. And in adding the staff that is needed. So what are the main goals of this exercise? We tried to put together five main goals of this exercise. First of all, to set up a baseline for the first review cycle. If there is a clear indicator coming from all the reviews that we are actually carrying out and the organizational reviews, is that ICANN cannot be compared to any other organization. So if we have to assess aspects such as effectiveness or efficiency of the activity of ICANN, the only way to do it is compare ICANN with ICANN. So we need to set up a first baseline study. And to observe, of course, best practices, also in other organization. But the baseline has to be set up in order to compare the following -- in the following exercises of review the performances of ICANN during the first cycle and during the following cycles. Then timely completion of the processes. We are obliged to respect the deadlines. We have these absolute deadlines of the end of 2010 for the first review, and then the other will follow. We want success of the evaluation exercise, but success is not enough. We want that the community perceive the success of the exercise. For this, we need buy-in from the community, and co-ownership of processes and outputs and outcomes of the exercise. As we said in the beginning, we are just starting some preliminary talks in order to pass to the drafting on the processes, but from the outset, it seems that we need, first of all, to put together a methodologically sound evaluation process. We need to define, to scope very well terms of reference. We don't want to put crimps in the terms of reference writing. We want consistent terms of reference, and related one each other. We need from community the availability of highly knowledgeable volunteers with available time. It's going to be an exercise that will imply use of resources from community member. We need properly representative review teams, of a small size. It will be very important for people to have available time and to be available to dig into data -- to read data, to interpret data, to summarize them, and to take conclusions. So it will be an operational task. And, finally, we will need more staff supporting the evaluation function. And suitable evaluation consultants. Peter, I will carry to you for -- >>PETER DENGATE THRUSH: Thank you, Marco. Well, that's the project. That's the challenge. What we need help with from the community is defining a number of those things that Marco has referred to, and that's the session we're kicking off today. I am going to hand over to Rod now to elicit from you assistance, comments. What should the terms of reference for those review teams look like. And all those questions. Over to you, Rod. >>ROD BECKSTROM: Thank you very much, Peter. And also I want to take a moment here to acknowledge some of the people in the room who did incredible work to make this thing happen on our side. Paul Levins. Paul, please raise your hands. He drove this in D.C. Theresa -- [ Applause ] >>ROD BECKSTROM: Big hand for Paul. Thank you, Paul. Is Theresa in the room? Perhaps -- and here is Theresa Swinehart. [ Applause ] >>ROD BECKSTROM: On the ICANN side. [ Applause ] >>ROD BECKSTROM: And Larry, Atlas and Fiona from Department of Commerce. They did an amazing job. Pleasure to work with. [ Applause ] >>ROD BECKSTROM: So what -- we really want to get your ideas and thoughts. I will talk just about some of the process of facilitation here, is when you want to contribute something, please come up to one of the mics here in the middle aisles so that people who are not in the room can hear you, and also the translators can hear you to translate into other languages as well as back into English if you are speaking in another language. And priority will be given to those who have not yet spoken. We want to encourage everyone to have an opportunity to share their thoughts. So if you have already spoken and someone else comes up who hasn't, please just let them step forward the front of the line. So let's open it up with some thoughts on process. So Marco has presented a framework here, overall, for beginning to think about this. What's some of the initial thoughts and responses you have? Marilyn. You are never shy. Pleased to see you up here. >>MARILYN CADE: Thank you, Rod. It's Marilyn Cade. I have actually the question that is minus one, before we get to question one. And then I will have more questions. In order to do an accountability review, one must first have accountability mechanisms. So we have, actually, an open public comment process on the establishment of accountability mechanisms. I was on the President's Strategy Committee and participated, as many members of the community did, as the full team did, in taking extensive input over a period of time that there was extensive dissatisfaction with the existing accountability mechanisms. That was reflected in the comments of the community, and the PSC report, supported by the board, called for major improvements. The proposal that's been put forward by the staff -- The PSC recommended a process for developing those accountability mechanisms, which the board did not follow, but -- yet. Always hopeful. But there is a draft -- there are two draft bylaw changes out for changes in the accountability mechanisms. The comment period closes, I believe, on November the 27th, Sunday after U.S. Thanksgiving. Now, in order to review, one must know what one is reviewing. How does the accountability -- the establishment of proved accountability mechanisms play into the timeline in process? And it's not a trick question. And had I thought it through, I would have asked it before. But.... >>ROD BECKSTROM: Thank you for the question. Peter. >>PETER DENGATE THRUSH: I get all the hard ones. Thank you, Marilyn. I think the reality is that you have highlighted the intersection of two separate processes. The President's Strategy Committee has been rolling in one way, and those resolutions that you refer to started off in -- at the Sydney meeting before the affirmation was even really very clearly contemplated. And so now we have got them both in front of us. The answer is I'm not sure exactly how we'll dovetail the existing development of new accountability mechanisms with an accountability review, but obviously you are absolutely right. That has to be done. >>ROD BECKSTROM: We now have the chair of the GAC, Governmental Advisory Committee, Janis Karklins here. >>JANIS KARKLINS: Thank you, my name is Janis Karklins, chair of the GAC. And I will peek without blessing of my peers in the GAC, because all what I am say now has not been endorsed by the GAC. And I am speaking as the chair of the GAC but without endorsement of my constituency. After publishing of the agreement, I realized that for the chairman of the GAC, that becomes more like a full-time job, a full-time volunteer job. And of course I tried to think through how to minimize the workload, because I'm afraid to be fired by my employer by doing my community service. But on more -- on more serious note, I have given some thought, and in my view, and where I'm coming from, is that these reviews is a secondary process of improvement of performance of ICANN which is outlined in commitments on the point 1.2.8, and review is secondary in this agreement. That's the first point. In my view, the review is not the review of the review teams, but that is a community review. And we need to construct the process in a way that community is involved from the very beginning to the very end in the review process. And I think this is something what is in the spirit of this agreement, how to do that. And here is the first -- what comes first. And in my view, first comes methodology. And if and when we will agree on methodology, it will be very easy to set up teams, because methodology will lead us to the question what is the composition of the teams, what is the size of the teams, and whether teams should be the same for all four of your processes or there should be different teams for different review processes. So Marco already outlined some steps which are classical for any review process, and I agree that the first thing would be to establish baseline or terms of reference or checklist, whatever you call it. But this is something against which we will make this review. Then this checklist should be published for public comments. And then after the period of public comments, it should be reviewed, and it should become a reference for review. And then I think that there should be two parallel data-gathering processes. One I would call the hard data gathering provided by the staff, by ICANN staff, which would be facts on all points which need to be reviewed for every time. And another is a community data, or soft data. It is impression how ICANN is performing on all of these check points. When this data is gathered, then there should be an expert who could make a synthesis, and the synthesis should be provided for the review team. And review team would match the hard data with perception of performance of ICANN, and based on that would draw conclusions and draft recommendations which then would be offered to the community for input. And after input, it would be taken into account, final recommendations would be drafted for submission to the board. Now, in my view, the work of review teams should be transparent to the extreme. And to the extreme, I mean they should be live video streamed or video cast. That everybody who wants to be present in the room should be present in live in the room. So that there shouldn't be any hidden moments when team does something that the community is not informed. In my view, this is the way how to ensure that there is no competition for being on these teams, because everybody will be in live present in the room in meeting. Of course, it will be difficult to allow everybody to speak. There are some limitations, physical limitations, but the mere fact of presence in the room live should facilitate decision of who should be on the teams. And agreement itself already provides some guidance. There should be representatives of each Supporting Organization or Advisory Committee. They are listed in the agreement. And I think that that would be the duty of each Supporting Organization or advisory team to propose those jewels who would be sitting on the review team and would do the work for that community, in full transparency. So this is my preliminary thoughts, and certainly I would love to hear criticism or support, most probably the latter most, on those proposals. Thank you. >>ROD BECKSTROM: Thank you very much, Janis. In fact, let me invite response to Janis's proposed framework. He is one of the parties, Janis and Peter as chairman of the board will be choosing the first review team members. I do want to invite comments on what he has. Do you have comments on what he has or a different issue? >>WOLFGANG KLEINWÄCHTER: No, no, I want to continue what he said. >>ROD BECKSTROM: Excellent. So let's spend some time on that and then we will move on to other topics as well. Please introduce yourself. >>WOLFGANG KLEINWÄCHTER: Yeah, my name is Wolfgang Kleinwächter. I'm from the University of Aarhus and chairing the next Nominating Committee. I've been involved in ICANN since the very first day. It's my 34th ICANN meeting. So I have followed this debate from various -- in various phases. And I think, really, the Affirmation of Commitments is a great step into still unknown territory. And we have to often explore ways, you know, how to review ourselves to find the right balance between internal and external review. All this is new and has to be tested out how it works. But one thing is for sure insofar I can only support what Janis has said just now: Transparency and openness is a very critical moment to avoid any impression that there is something done behind closed doors or any conspiracy or just among friends or friends' networks. I think in one of your slides, there was a contradiction when you said small size with few panels, but representative. Because you have so many representatives in ICANN, so many constituencies, that it's rather difficult, you know, to be representative and small at the same time. So how to settle this? And I think Janis made already a proposal in a certain direction, and I will push this a little bit forward. Supporting organizations and advisory committees can send people to this review committee, or should compose the review committee. So there could be some outside or in a different direction, that probably the person nominated by the advisory committee or the supporting organization organizes in its own way a communication, you know, within the constituency, so that he is used as a channel. So that means that more people are involved, that you have like a pyramid, a system where, you know, on the first layer, you have a small-sized group. But more people are involved which have a permanent communication. And this will bring a lot of transparency and also inclusion to the process so that people know they have a channel if they want to say something, if they want to make a contribution, here's the channel. And then you have a multi-layered system with the small-sized review team on the top. Thank you. >>ROD BECKSTROM: Thank you very much. Milton. >>MILTON MUELLER: Yeah. I deliberately let Wolfgang go first, because I thought he was going to speak more directly to what Janis had just said, and I think he did. I want to go back to Marilyn's question. And it's not too often that Marilyn and I are sort of singing the same tune, so I think it's -- I still feel that I can't answer questions about methodology until I have a better grasp, a better concept of what these reviews are supposed to do. The interesting thing about your response to Marilyn, Peter, was that it seemed like, in your mind, you put accountability in a separate box from these reviews. And maybe that's correct and maybe it's not. But I thought that these reviews were about accountability in some way, and therefore I have basic fundamental questions, like, when a review panel reviews ICANN's performance, is it assessing the degree to which it deviated from its bylaws? Or is it assessing the degree to which it didn't do something that the community wanted? Is it second-guessing, in effect, the board? When it makes the recommendation, what does the board do with that? It's not binding. That's kind of clear. It's not exactly an edict, like a court. So what does -- what happens then? And the reason that's -- Let me just give you a very concrete example, okay? We go through, let's say, some kind of a policy development process on security, and it takes two years, okay. And, hopefully, your two-year policy process intersects with your review process such that the review process begins after the policy process has ended, which may not happen. And then does the -- then you're talking about a very elaborate process. In some ways, we like what he proposed. But then you're going to go do it all over again; right? You've involved the community in making the policy. Then you're going to involve the community in reviewing the policy. The two communities seem to be fundamentally the same things, the same people, the same supporting organizations. So I'm just confused about what the object is, other than the consumption of our endless resources of time and energy, which, you know, we all would devote to ICANN if we could, but sometimes there are, indeed, limits on what we can do with our time. So.... >>ROD BECKSTROM: The words are there for all to absorb in the Affirmation of Commitments. And there's a lot of interpretation to be done. And I think -- so I hear, Milton, you asked some definitional questions. What is meant by "accountability." So it sounds like you're suggesting that's a question that review teams should first address; right? >>MILTON MUELLER: So you're saying this is an accountability mechanism? >>ROD BECKSTROM: I heard you say that you thought there is different ways to define "accountability," and is it accountability of following the bylaws or is it accountability of the board making decisions. I was just saying, there's a definitional question that you posed to the review team. And so I'm just highlighting that as part of what I'm hearing you said in restating it that, for the group to consider, is I'm saying it's not for me to answer. I'm not even on the initial review team, presumably. And -- okay? So -- and the members are yet to be appointed. So I think you raised some really good questions. And I was just inviting others to come up when they wish to help share their views on those answers. >>STEFANO TRUMPY: Stefano Trumpy. Is it running? >>ROD BECKSTROM: It should work if you get close to it. Can you speak -- >>STEFANO TRUMPY: -- >>ROD BECKSTROM: Can you move to this one quickly? AV -- >>PETER DENGATE THRUSH: Stefano, Milton, another just high-level extract from yours is the objectivity, having done the work, then the same people are going to be reviewing the work is a problem. We're going to have to make sure the reviews are objective. >>STEFANO TRUMPY: So Stefano Trumpy, GAC, Italy. And I'm speaking as a single GAC member, of course. So the question of Milton was a nice question. How quick we could expect. And certainly these four reviews saw a great achievement also in the direction of the WSIS document. Because ICANN used to be -- used to report to a single government, and now it's reporting to the community. So -- and this is going to realize something. And let me start from this to say that the process, the famous enhanced cooperation that was one of the results of the Tunis Agenda was not activated so clearly in the beginning. So in the first IGF in Athens, ICANN was not an argument for discussion. It was not inserted into the critical resources as an argument of discussion. But starting from Rio, then every year, there was the chance for ICANN to demonstrate how the internationalization of the management of the DNS is progressing. And of course it's not something similar to the reviews. But ICANN had the occasion to demonstrate to a large public. And the large public is formed by inside -- let's say by people connected to the ICANN process -- and also by those that do not agree about the ICANN model. And we have the occasion to exchange views. And perhaps this was a way to at least demonstrate that it was not so negative, let's say, for the non-friends of the ICANN model. So doing such kind of exercise is certainly an occasion to be even more convincing, let's say. But we have to realize that there are still a group of people that are not in favor. And they already started to make some comments, saying, "Look, now ICANN is making out reviews, is making something that is organized by ICANN itself just to demonstrate to us that we do not like such a model that they are good guys." So I think that the -- in the panels, maybe we should have people that are maybe critical, but not exactly on the other side. While with the transparency that was recommended by our chair, then, of course, the panels will receive also a bunch of negative comments. But in the end, this is the good of democracy, let's say, to let people talk, and then, in the end, the panel will have to make some summary and eventually provide also some criticism into the ICANN board. So it's a great occasion. We have to balance too much internal comments to the -- also those that have critics. So in this way, we can reach what I like to call the external accountability of ICANN. Thank you. >>ROD BECKSTROM: Thank you very much, Stefano. Bertrand. >>BERTRAND DE LA CHAPELLE: Hi, my name is Bertrand de la Chapelle. I'm the French representative in the GAC and the vice chair of the GAC. I'd like to walk a little bit in the steps of Marilyn for one second, because, actually, I sense among all the discussions -- and it's sometimes the case in other spaces -- a sort of immediate focus on the reviews, then the first review, and then the composition of the team. And I would like to support what Janis was saying. The Affirmation of Commitments is basically about commitments. And interestingly enough, if you look at the content, I would expand a little what Janis said in paragraph 1 to 8. I would add that each of the paragraphs in 9, the beginning is the commitment. In 9.1, for instance, you have the five items that are actually here. And so the reviews are only the end of each of paragraph 9. These commitments are actually, in the case of the first subject, matching very nicely the internal bylaw reviews or the internal reviews mandated by the bylaws. The first one is actually on the evolution of the board. There is a board review process. There's another one on the evolution of the GAC. There is now a joint GAC-board working group that deals with the role of the GAC. There is the mention that is not here but is in the affirmation of commitment of an appeal process. It's part of the IIC. And I will go on like this. The first point is to make the very strong point that ICANN is currently undergoing a certain number of reviews under the coordination of the SIC, but beyond, that have to deal a lot with the building of the accountability mechanisms that this is going to review. And so could we focus, please, first during the few months ahead on the existing, ongoing reviews that are producing bylaw revisions that we should analyze in a coherent and holistic manner, because they interact with one another? So the first consequence from the Affirmation of Commitments that is a very good document is that we should focus on the accountability mechanisms improvement during the coming months. The second point, and I am shortly done here, is that with due respect to the work that has been done by Marco, I'm a little surprised at the chart that has been presented. And I'm wondering whether we're not overcomplicating something that could be relatively simpler. The reason why I say that is because the first review mechanism is basically replacing what the United States government was doing, basically, every three years. What the United States government was doing every three years, or sometimes in the midterm review, was relatively simple. Notice of inquiry to the whole world; consultations here and there; and a summary that was relatively succinct, because in most cases it was just the update of the memorandum of understanding. I do not, -- I hope the mechanism we're putting in place is not going to lead to what we have here, like the whole process to develop the term of reference. I'm sorry, the term of reference, they are here. You take the paragraph. You're done. The Working Group on Internet Governance, after the first phase of WSIS, had a mandate which was basically five lines or six lines. And it produced a real good amount of work. You don't need anything more as a term of reference. You don't need external consultants as the main workhorse, and more than anything, beyond the requirement for a very open and transparent process. What we need is a report, in the end, that is drafted and written by the team with broad consultation and that is short. If the Affirmation of Commitments could produce such a major change with a three-page document, I hope that this review will not occupy all our energy in the next nine months or ten months and produce a 40- page document. [ Applause ] >>ROD BECKSTROM: Well put. Very well put. And first I just want to say thank you for any suggestions on simplification. And I think you had two major ones. One proposal with respect to integrating some of the bylaw reviews, potentially with these other reviews, which is worthy of examination, as well as a proposal on terms of references. So thank you very much, Bertrand. Peter, do you have any comments? So next person, please. >>BECKY BURR: Becky Burr. Broken record on this topic. Peter, you know what I am going to say. Marilyn said before we can review the accountability, we need to have accountability mechanisms in place. I have a really -- I want to make this as concrete as possible. The President's Strategy Committee spent 18 months -- I don't know, an enormous amount of time working on the improving institutional confidence provisions. It had one really important recommendation, which was appoint an experts committee to come to this organization with proposals for accountability mechanisms. That was a fabulous proposal. What happened? I don't know. That went away and instead we got two precook recommendations that are now out for public comment. To get to a place where the accountability review board can do its work, the first step I would take is implement the recommendations of the President's Strategy Committee and appoint an experts committee to come back with real proposals for real accountability mechanisms. >>PETER DENGATE THRUSH: Thank you. >>JONATHAN ZUCK: Yes, my name is Jonathan Zuck and I am with the Association for Competitive Technology and a fellow broken record, I suppose. I think part of the difficulty here in determining a methodology for review is that it's too much of an abdication of responsibility to the review committees. And for there to be an adequate review that's fairly streamlined, I think there has to be more of a management methodology in place in the first place. And I think clear and precise strategic objectives and goals that can be measured both in terms of the current state of different issues that are mentioned in the strategic plan, the objectives for improvement of those states, and then the measurement of the success of those improvements will go a long way to streamlining any kind of a review process. I mean, we look at the -- just an example of something like compliance. Right now the only two metrics we have is how many responses were handled and how many people were discredited, or de- certified; right? And so there is nothing about percentages of people that were -- of issues that of were raised in a certain period of time or that were under the jurisdiction of ICANN and things like that. There are plenty of concrete numbers that can be used to really measure both the strategic objectives of the organization and the success of those objectives. I think if we do that, we will go a long way toward making the process of reviewing ICANN much simpler because we will have objectified the process, at least to the degree possible. >>ROD BECKSTROM: I think those are very productive suggestions, particularly defining some clear metrics which clearly the review -- Marco, please. >>MARCO LORENZONI: With just an additional remark, which links up your intervention with Milton and with Janis's intervention. In this review the focus will not be only on quantitative measures but also qualitative measures. And this will be the very challenge. I will give you an example. An organization can be considered the most transparent organization in the world because it publishes everything, everything that is produced by the organization. And so it can be quantitatively assessed as transparent. But if the community perceives this organization as opaque, then the organization has a problem. And so it is only from the matching of, in this case, interviews, for instance, with community members and with the larger Internet community and the factual data that we can arrive to drive conclusions on transparency, for example. Just one example among the many we can give. >>JONATHAN ZUCK: Can I respond very quickly? I think even nonquantitative factors can be, in some instances, quantitatively measured, though. If you make start to make more formal surveys of the community, et cetera, to actually understand where things stand, where confidence stands. I mean, I am very concerned that there's no session at this entire meeting about improving institutional confidence. That somehow seemed to get rolled into something that isn't directly related to that, and it was a very important initiative I think the organization was going through. And I am concerned it sort of got transitioned out by what should be a fairly limited replacement for the U.S. government role in ICANN's operations. >>ROD BECKSTROM: Thank you. Sir. >>NARESH AJWANI: Naresh Ajwani. I am a director with Network Internet Exchange of India. AoC has generated a very good feel factor across the entire globe, and I am worried with the mindset I have heard until now, it may not get mitigated or challenged by not letting the entire process laid out here. When I see this process, areas of analysis, I feel ICANN wants to involve more and more people. Definitely that means reviews. But definitely, it also means engagement of the people who are not engaged until now. And I don't think they have no right to get engaged with this kind of a scenario which is imaging out that we should check it or we should control it. I personally feel the ecosystem of Internet shall be expanded. More people shall be involved, more consultants shall be there, and more review shall be there. Yes, there will be delay. But definitely, it won't be wrong. Please take this into consideration. >>ROD BECKSTROM: Thank you very much. And what I will say is two points. First, no one is trying to control this. That's why we are having an open and public session, and the review teams will be helping to define this. And, secondly, please recruit new candidates to the community and introduce candidates to the two selectors of the review teams that are in the room, which is Peter and Janis. >>NARESH AJWANI: I am getting worried. I am finding people not -- (speaking off mic). >>ROD BECKSTROM: Welcome to the multistakeholder community. There's a rich variety of opinions and some consensus will hopefully be found by the review team in what they are doing. Thank you. >>WILLIE CURRIE: My name is Willie Currie. I am from the Association for Progressive Communications, a member of the NCUC. This is my first ICANN meeting, and I just want to make an observation, perhaps with fresh eyes on the ecosystem concept that is very visible here. It seems to me that while I take Bertrand's view that a review should be short and to the point, there may also be value in trying to probe what it is about ICANN as an ecosystem that works or doesn't work. And in that context, I would suggest that perhaps an anthropologist is hired to have a look at the culture of ICANN, what it is about this multistakeholder structure with its various supporting organizations and advisory committees and how they relate to the board. I say this partly because coming into this week-long meeting for the first time, what is apparent to me is that it seems that ICANN is in some ways organized vertically. The Tuesday meetings, for example, are all separate constituency or stakeholder or advisory committee meetings. And that if one were to look at this from a kind of ecosystem or anthropological perspective, one could say where are the horizontal connections between these different structures, and is there something in this ecosystem, by being vertically organized, that means that everyone is competing or jostling for the attention of the board. And what does that dynamic then create within the board as it looks at all of these constituencies within this community that are trying to get the attention of the board. I think there was only one horizontal meeting that I could see between the GAC and ALAC. >>ROD BECKSTROM: This might be another.... >>WILLIE CURRIE: Right. So it's just that sense that perhaps, yes, a quick review on performance and whether the targets are being met is one thing, but the other is actually looking at what is specific about this ICANN as an ecosystem. >>ROD BECKSTROM: Sure. Thank you very much. I am personally am fascinated about your idea of bringing in some anthropologists, because my experience is the sociological anthropological dimensions of human systems is vitally important and often overlooked and underappreciated. So point well taken. It will be up to the review selection chairs to determine whether they seek such an expert for the review team. Irrespective of that, as the CEO of ICANN, I am very interested in that idea. If you know any volunteers or even professionals to potentially look at. With respect to the meeting structures and the verticality, which is a real issue, also given how the board members are elected by vertical constituencies and then a mixed NomCom cross-community constituency, an interesting issue. But we are flexible. We should run these events to be, you know, evolutionary and changing over time. That's one of the reasons why we brought the musician out on Monday morning, is just to kind of bring in a new element. So we want to have a new ICANN as we move forward here. So well taken. There will have to be some tradeoffs then in terms of giving more time to horizontal pieces than vertical, which is certainly what you are indicating as a position. I don't remember who is next, so I will let you two decide. >>PAUL FOODY: Hello, welcome to ICANN, Mr. Beckstrom. Paul Foody. Gianni Agnelli famously said to be successful an organization needs an odd number of members, and three is too many. So with 7 billion people to answer to, I don't think many people really envy you your task. But the points about transparency, those people deserve to be given as much advice and information about what is going on as possible. The ICANN has the WHOIS list. It's a list of all the domain registrant owners. >>ROD BECKSTROM: A correction. We do not have the WHOIS list. We set the policies for the WHOIS list that are held by registries. >>PAUL FOODY: Okay. ICANN does not -- >>ROD BECKSTROM: Registrars. Sorry. Thank you. >>PAUL FOODY: ICANN does not hold a global monopoly on trademark law. And yet part of the new gTLD process virtually rewrites that. So given the fact that when I, as a registrant, get into a problem with my registrar, I revert to you guys, maybe job number one for ICANN is to establish and create -- you know, change the bylaws so that you can communicate directly with at least the domain name registrants. For me, in terms of transparency, that has got to be job number one. And the fact that that's not in place, it's not good enough just to have the excuse. >>ROD BECKSTROM: Sir, thank you very much. And I think it's a fine suggestion to drive into one of the policy processes. I think the topic we're trying to review today is how the review teams will coordinate themselves. And they might criticize, such as your thoughts might be incorporated, but you really have a policy proposal, and we would fully encourage -- >>PAUL FOODY: Also, there's a transparency thing. >>ROD BECKSTROM: Right, a proposal that would lead to more transparency. But it's a policy change in, I think, how ICANN works. Thank you very much. >>WERNER STAUB: My name is Werner Staub. I have two very small questions. One of them is, why this session here is not being transmitted in realtime, neither by the scribe feed nor by audio? >>ROD BECKSTROM: Not that I know of. >>NANCY LUPIANO: It was requested by one of our staff members. >>ROD BECKSTROM: Okay. Can we turn on the scribe live to the world right now, please? If not, the video cast? Okay. All right. >>WERNER STAUB: The other question was also a very short one. Is there a particular reason why, in the Affirmation of Commitments, the definition of "DNS" is different from anything that people would usually use? [ Laughter ] >>PETER DENGATE THRUSH: They -- The negotiations around this are confidential. But let me tell you that people who are aware of those negotiations are laughing because it's exactly the problem -- The answer is, we had to go back to an older set of words. So we got around that by putting a footnote to explain what we mean by DNS. It was done at a reasonable pace. There were a number of editing suggestions that could be improved. And I think you might have touched on one of them. >>ROD BECKSTROM: Thank you. And you win the proofreading award. >>PETER DENGATE THRUSH: While there's nobody at the microphone, perhaps I could just respond a little bit about some of the things. Clearly, the work -- well, in my view, it's clear that the work in relation to this, the primary, the big work, is going to be in setting the standards. That's really the role for the community, it seems to me. We need to get community input into what those standards should be. And then we need community buy-in to the compromise that we eventually will construct. And then we need sign-off on that so that we can then start putting the teams to work. The actual review teams themselves, in my view, should be small and lightweight, which I think is the recommendation we heard yesterday from the GAC. I'm really seriously thinking there will either be one representative of each of the classes. That gives us a team of eight or nine. As soon as we go to two, we've got 20, which, of course, becomes about five times harder. If we have three, of course, we've suddenly got half of ICANN, half of a constituency. So they need to be small, and they need to be lightweight. And the work that they do should really be a relatively easy, quick check of performance against the standards that have been created. So if the standards have been created carefully and properly, then, with a bit of extra assistance from some consultants or even the volunteers -- and, remember, we can appoint experts -- then the check should be quite good. But the other part of this, the real work -- and this is perhaps a slight answer to Mr. Zuck -- the real work, the reason why we are looking at these particular areas, is because they've been identified as needing work. We're not asking to report on the things that ICANN does really well. These are the things that, through the institutional -- improving institutional confidence exercise and the joint -- the midterm review of the joint project, the community said we need to improve these things. Now, we had hope that improvement work was all going to be done by the time we got to the end of the JPA. But other things interfered. Some of you think we need new gTLDs, and some of you think we need new IDNs, and there are other distractions from this particular work. But that is the exercise. These are the things that we need to be working on. So this is perhaps coming -- circling right back to sort of Marilyn's view in relation to those accountability mechanisms. What we need to say, is -- and, Becky, I agree, we need to build new accountability mechanisms. So we start building those, and then the review will come along and say, "Let's review. Have we got set up accountability mechanisms?" In the first review, we may not have finished building them and testing them and cycling them. So the review will say, "No. We see there's work under way." But at this stage, of course, if we can finish that work, get those accountability mechanisms in place, then the review team when they look at accountability will say, "Yep. We now have an ombudsman. We now have a this. We now have appeal. We now have board" -- you know, all the mechanisms that we currently lack. If they're in place and time, then that review will give us a tick. And, of course, we've got to do this again and again and again. And these are the sort of things that ICANN needs to do. >>ROD BECKSTROM: Thank you. I think we're actually at our time limit. We had one hour. And we're at the limit. So I want to thank everyone very much for all of your contributions. Thank you. [ Applause ]