octavia failed to run
preface
If not specifically noted, all command lines are executed using the admin account.
Problem description
Create a loadbalance by calling octavia's api as follows
@token = gAAAAABe6y8up_ZdUS43Lfe8AFERncGtZFK6IHrnd-MSgbE5U-xKg-6N04eUV3tNwlKEAiZVNIl6I0NVvjMT97CygiJL-7XFciQHl0vWdd2rTQWtaDR9w04pJtk1DYBi7vjcq0VivngYMrn2MhOJmf3X3Fss9kV2BY9m19v-A6mba9CYVPE_NHY
@endpoint = http://10.8.8.11:9876
###
# create loadbalancer
POST {{endpoint}}/v2.0/lbaas/loadbalancers
X-Auth-Token: {{token}}
Content-Type: application/json
{
"loadbalancer": {
"description": "loadbalancer_created_by_gxj_for_test_purpose",
# "flavor_id": "30ad863a-25dc-425a-8ebe-e29f361cb777",
"admin_state_up": true,
"vip_subnet_id": "3aa77344-3086-4d2c-aa1a-27c997c92840",
"provider": "octavia",
"name": "loadbalancer_created_by_gxj_for_test_purpose"
}
}
Successfully returned after calling
HTTP/1.0 201 Created
Content-Length: 681
Connection: keep-alive
Content-Type: application/json
Date: Thu, 18 Jun 2020 09:34:14 GMT
keep-alive: timeout=4
Proxy-Connection: keep-alive
Server: WSGIServer/0.1 Python/2.7.12
X-Openstack-Request-Id: req-db7143a3-d2aa-450e-8414-1b2fa257e974
{
"loadbalancer": {
"provider": "octavia",
"flavor_id": "",
"description": "loadbalancer_created_by_gxj_for_test_purpose",
"provisioning_status": "PENDING_CREATE",
"created_at": "2020-06-18T09:34:14",
"admin_state_up": true,
"updated_at": null,
"vip_qos_policy_id": null,
"vip_subnet_id": "3aa77344-3086-4d2c-aa1a-27c997c92840",
"listeners": [],
"vip_port_id": "34f53806-1c12-48a5-8724-d965cb74cf7e",
"vip_network_id": "0c1c8b1d-6515-4e5c-bba2-4a5e3ed80084",
"vip_address": "10.0.0.29",
"pools": [],
"project_id": "55982e2c523149158a6cc839ce2eece2",
"id": "59b14489-3f4f-49dc-8533-29fc50377eb9",
"operating_status": "OFFLINE",
"name": "loadbalancer_created_by_gxj_for_test_purpose"
}
}
But after a period of time to check the status of the loadbalancer, it was found that it was an error.
Troubleshooting
General process
- octavia's general workflow
- View the log of Octavia API
- View the log of Octavia controller worker
octavia's rough workflow
The picture is official
Trust reference OpenStack Docs: Octavia v0.5 Component Design
View the log of Octavia API
Via vi /var/log/octavia/api.log Query the id of loadbalancer and find the following logs
2020-06-18 17:34:14.880 24792 INFO octavia.api.v2.controllers.load_balancer [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Sending created Load Balancer 59b14489-3f4f-49dc-8533-29fc50377eb9 to the handler
It is found that the creation request from the client has been sent out to the controller worker. That is to say, octavia is OK, but looking up, we find some error s
2020-06-18 17:34:03.665 24792 DEBUG octavia.common.keystone [req-72f1be5c-a7e3-42bc-9590-032ec02a1b0a - 55982e2c523149158a6cc839ce2eece2 - default default] Request path is / and it does not require keystone authentication process_request /usr/lib/python2.7/dist-packages/octavia/common/keystone.py:67 2020-06-18 17:34:13.943 24792 DEBUG octavia.network.drivers.neutron.base [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Neutron extension security-group found enabled _check_extension_enabled /usr/lib/python2.7/dist-packages/octavia/network/drivers/neutron/base.py:66 2020-06-18 17:34:13.950 24792 DEBUG neutronclient.v2_0.client [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Error message: {"message": "The resource could not be found.<br /><br />\nExtension with alias dns-integration does not exist\n\n", "code": "404 Not Found", "title": "Not Found"} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:259 2020-06-18 17:34:13.951 24792 DEBUG octavia.network.drivers.neutron.base [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Neutron extension dns-integration is not enabled _check_extension_enabled /usr/lib/python2.7/dist-packages/octavia/network/drivers/neutron/base.py:70 2020-06-18 17:34:13.957 24792 DEBUG octavia.network.drivers.neutron.base [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Neutron extension qos found enabled _check_extension_enabled /usr/lib/python2.7/dist-packages/octavia/network/drivers/neutron/base.py:66 2020-06-18 17:34:13.965 24792 DEBUG octavia.network.drivers.neutron.base [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Neutron extension allowed-address-pairs found enabled _check_extension_enabled /usr/lib/python2.7/dist-packages/octavia/network/drivers/neutron/base.py:66 2020-06-18 17:34:14.018 24792 DEBUG octavia.db.repositories [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Checking quota for project: 55982e2c523149158a6cc839ce2eece2 object: <class 'octavia.common.data_models.LoadBalancer'> check_quota_met /usr/lib/python2.7/dist-packages/octavia/db/repositories.py:313 2020-06-18 17:34:14.047 24792 DEBUG octavia.network.drivers.neutron.base [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Neutron extension security-group found enabled _check_extension_enabled /usr/lib/python2.7/dist-packages/octavia/network/drivers/neutron/base.py:66 2020-06-18 17:34:14.052 24792 DEBUG neutronclient.v2_0.client [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Error message: {"message": "The resource could not be found.<br /><br />\nExtension with alias dns-integration does not exist\n\n", "code": "404 Not Found", "title": "Not Found"} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:259 2020-06-18 17:34:14.052 24792 DEBUG octavia.network.drivers.neutron.base [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Neutron extension dns-integration is not enabled _check_extension_enabled /usr/lib/python2.7/dist-packages/octavia/network/drivers/neutron/base.py:70 2020-06-18 17:34:14.057 24792 DEBUG octavia.network.drivers.neutron.base [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Neutron extension qos found enabled _check_extension_enabled /usr/lib/python2.7/dist-packages/octavia/network/drivers/neutron/base.py:66 2020-06-18 17:34:14.063 24792 DEBUG octavia.network.drivers.neutron.base [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Neutron extension allowed-address-pairs found enabled _check_extension_enabled /usr/lib/python2.7/dist-packages/octavia/network/drivers/neutron/base.py:66 2020-06-18 17:34:14.070 24792 DEBUG octavia.network.drivers.neutron.base [req-db7143a3-d2aa-450e-8414-1b2fa257e974 - 55982e2c523149158a6cc839ce2eece2 - default default] Neutron extension project-id found enabled _check_extension_enabled /usr/lib/python2.7/dist-packages/octavia/network/drivers/neutron/base.py:66
The error should be that the requested resource 55982e2c523149158a6cc839ce2eeece2 does not exist. I felt it was a network related resource, so I went to the following command
root@ctl01:~# openstack network list | grep 55982e2c523149158a6cc839ce2eece2 | 26aef702-6d8e-4384-9c87-5d7c851021ac | HA network tenant 55982e2c523149158a6cc839ce2eece2 | 022fb016-e250-42ad-95e2-8ba4f26350d5
In fact, this uuid is a tenant, which is often unexpected
So I tried
root@ctl01:~# openstack project list | grep 55982e2c523149158a6cc839ce2eece2 | 55982e2c523149158a6cc839ce2eece2 | admin | | 9d99f566b76246d4b72722b9882c245f | 55982e2c523149158a6cc839ce2eece2-b77e26e9-a4a6-4773-a42d-7ef029d |
It seems to be the project of admin_ ID. however, it's not used by software. When you look at the log, you find that you have missed the most important thing nextension with alias DNS integration does not exist. This should be neutron dns as a service under Google
View the log of Octavia controller worker
Open the log file "/ var / log / Octavia" of controller worker with vim/ worker.log "Find useful logs based on the id of the loadbalancer that failed to be created last time. Here are the related logs.
1223 2020-06-18 17:34:14.922 3374787 INFO octavia.controller.queue.endpoint [-] Creating load balancer '59b14489-3f4f-49dc-8533-29fc50377eb9'... 1224 2020-06-18 17:34:14.984 3374787 INFO octavia.controller.worker.tasks.database_tasks [-] Created Amphora in DB with id ee934942-fc30-44ef-b834-e2dda0cd650f 1225 2020-06-18 17:34:15.031 3374787 INFO octavia.certificates.generator.local [-] Signing a certificate request using OpenSSL locally. 1226 2020-06-18 17:34:15.032 3374787 INFO octavia.certificates.generator.local [-] Using CA Certificate from config. 1227 2020-06-18 17:34:15.032 3374787 INFO octavia.certificates.generator.local [-] Using CA Private Key from config. 1228 2020-06-18 17:34:15.032 3374787 INFO octavia.certificates.generator.local [-] No Passphrase found for CA Private Key, not using one. 1229 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver [-] Nova failed to build the instance due to: Network 8a588b4c-06ae-4ba2-ae74-2a 8a6ed687ef could not be found. (HTTP 400) (Request-ID: req-d8b979fe-0cc5-4b3a-a1aa-3d7e457ca8c0): BadRequest: Network 8a588b4c-06ae-4ba2-ae74-2a8a6ed687ef could not be found. (HTTP 400) (Request-ID: req-d8b979fe-0cc5-4b3a-a1aa-3d7e457ca8c0) 1230 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver Traceback (most recent call last): 1231 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver File "/usr/lib/python2.7/dist-packages/octavia/compute/drivers/nova_driver.py" , line 154, in build 1232 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver availability_zone=CONF.nova.availability_zone 1233 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 1313, i n create 1234 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver return self._boot(response_key, *boot_args, **boot_kwargs) 1235 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 772, in _boot 1236 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver return_raw=return_raw, **kwargs) 1237 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 366, in _crea te 1238 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver resp, body = self.api.client.post(url, body=body) 1239 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 310, in post 1240 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver return self.request(url, 'POST', **kwargs) 1241 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 83, in requ est 1242 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver raise exceptions.from_response(resp, body, url, method) 1243 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver BadRequest: Network 8a588b4c-06ae-4ba2-ae74-2a8a6ed687ef could not be found. (HT TP 400) (Request-ID: req-d8b979fe-0cc5-4b3a-a1aa-3d7e457ca8c0) 1244 2020-06-18 17:34:15.561 3374787 ERROR octavia.compute.drivers.nova_driver 1245 2020-06-18 17:34:15.561 3374787 ERROR octavia.controller.worker.tasks.compute_tasks [-] Compute create for amphora id: ee934942-fc30-44ef-b834-e2dda0cd650 f failed: ComputeBuildException: Failed to build compute instance due to: Network 8a588b4c-06ae-4ba2-ae74-2a8a6ed687ef could not be found. (HTTP 400) (Req uest-ID: req-d8b979fe-0cc5-4b3a-a1aa-3d7e457ca8c0) 1246 2020-06-18 17:34:15.561 3374787 ERROR octavia.controller.worker.tasks.compute_tasks Traceback (most recent call last): 1247 2020-06-18 17:34:15.561 3374787 ERROR octavia.controller.worker.tasks.compute_tasks File "/usr/lib/python2.7/dist-packages/octavia/controller/worker/tas ks/compute_tasks.py", line 99, in execute 1248 2020-06-18 17:34:15.561 3374787 ERROR octavia.controller.worker.tasks.compute_tasks server_group_id=server_group_id) 1249 2020-06-18 17:34:15.561 3374787 ERROR octavia.controller.worker.tasks.compute_tasks File "/usr/lib/python2.7/dist-packages/octavia/compute/drivers/nova_ driver.py", line 160, in build 1250 2020-06-18 17:34:15.561 3374787 ERROR octavia.controller.worker.tasks.compute_tasks raise exceptions.ComputeBuildException(fault=e) 1251 2020-06-18 17:34:15.561 3374787 ERROR octavia.controller.worker.tasks.compute_tasks ComputeBuildException: Failed to build compute instance due to: Networ k 8a588b4c-06ae-4ba2-ae74-2a8a6ed687ef could not be found. (HTTP 400) (Request-ID: req-d8b979fe-0cc5-4b3a-a1aa-3d7e457ca8c0) 1250 2020-06-18 17:34:15.561 3374787 ERROR octavia.controller.worker.tasks.compute_tasks raise exceptions.ComputeBuildException(fault=e) 1251 2020-06-18 17:34:15.561 3374787 ERROR octavia.controller.worker.tasks.compute_tasks ComputeBuildException: Failed to build compute instance due to: Networ k 8a588b4c-06ae-4ba2-ae74-2a8a6ed687ef could not be found. (HTTP 400) (Request-ID: req-d8b979fe-0cc5-4b3a-a1aa-3d7e457ca8c0) 1252 2020-06-18 17:34:15.561 3374787 ERROR octavia.controller.worker.tasks.compute_tasks 1253 2020-06-18 17:34:15.563 3374787 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-cert-co mpute-create' (2f7f05b0-c7a3-4fa1-94f2-584c70ca20d1) transitioned into state 'FAILURE' from state 'RUNNING' 1254 8 predecessors (most recent first): 1255 Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-cert-expiration' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'server_pem ': 'gAAAAABe6zUXLP9ofIXoOlDm1_OLc5arzZh_Bo2aunp_iGB9qiRS8X-O79tIYMY24-zhg-MD8JBpWQsZ9hEQsEFYLKaVKjSKUNX9oRkjeyWLZXeAyKhYJ6GN6tndes2LMSL0GrfHqs27fg2UlYuQ0F cLMpfEKoLXLLc7rqIsHdk1bMTIc0uMAAPE4b4QPPH3G0ta5Mk5uBxrrf5lgNAusjdy4Xygvzu5kXlGXrov50sVdQIINGIbq9GwL6hJYG2SHHz0w2-m0EAG6i_31zDYL76D5gVcCVSt0-pKFOdydOe1am23 Gels0RunQgXhNnQ6yiZJf0L1T9XsrkyFP9u6QmCe5cGF48u0Ddgaa9qOXNuOtLLlWFpkmMQbHOOpDMt96w-3ra149J4R39HMQc8DJdBhE5m-x6XxbmCzojPrFtbO0cyGn3pnkhClPdgsGZtuw5rUFm5SQu Yw76HuGB2vxe5qeN6JVccGHhJJUum4XRZHmgJEV7kkxeYz0JYwM2TUDYOdOxten3Jb55-YoaB4X0dcamdx6InFYJHueG8VH7l0BPk5OjxHzyWsOfj5cFiLlng3MdqWa3Lw_zUYACF9HHkUycHJ1v-NNzWf VCFo-8yaDpb6I1tCtwrr6fBdv0x0O_L0XTRNnHS2uztHneIFcXbAfj0gWz1Y2yhpJtLR69PgOM_Qg6RcAZ1fFv4QBynPJ2Zr1D6wJKgz0yJygHrW4LGYrOdnHPHugIKn38xP68qzBNpg-BCqNsjmepPJvx cJXh9XVrvFKyYkrr7uD5OGuo1yHAXXK4Ur29H9-O4qMZICllHWs7tXa3gKKLqt2jXHYheeIoOwC49bieaLw4RLVuKaQoxrHcqezYeRCtuUo0azZuqP3zdTjO_ywGniw1U0zL5Un5-siO1ukqiadAQOByTT gd7hpjEl6CW85sukEjZRSOpDMikg1uodBN4ewTZa_J-u4ujEJaMwEnnjqepwXpfQ0Qbzq3ZtXq-g1_q9GJttYGEtyJqTioq6YS5jSUXX4d039NjtsLU0b_voEbXBon6bwZdCTnUEnHQ7kNqdA_Zm1oQMZX hbAhRWc_chzXx_75TszeRumBwo5DJbmfxuj4BG7IQgtuY-26gH8TZEVVGDNvfNIKXZD-faWR9UXw5KU3SpKU1QYGM5r1oRypRH0NUka4pk-pJzwBpfWZg3gy1-yFmblyGr441-Pfm2cWSXSP0f1oB2mhNf WJWwKGCKD3f27q5E5ln4guOwPHgg4Ioy3uY8QwfQz_LKlY3UmGr-BMsfi2cfxYru76fAKxUf1V5nL4Bn4Lg2a_pa2y2d2c6v2JBWZZnmwR-yXO1Mxq4e8iock-2FNVlHZuRi2xwNPWn5TqlDaqoxbYmPpD Deonecix7Q2po_TWjFnGHAMZzX0qka26Sm4yd6JKtIdfakhqakTma_EqmxM2L9HkldLbljCkn9Q8tLsiv4G4x1lz8ZSETjNCflZmuGjLUbXBVCEQl2I398j-XOZ4F1MBSc6hppg9_MwzF17i8w29T3PxUd jNjA47e1DCb-tD8NnF43BuLufkOzzeRBs5CBXso9SzOTJMtA74EeVzMsDRaFyjZGdWerfKqNisfpzreM1pgLRaM_4BcqPOg5OcaGHrWHhGUV0ov5dzfkzPqskHS-KeI2nT5G3jl37QGfNleN6oBkOArkJz RJ60Q8xlnqjGex89wClAV_dWKVs7XoVxdOvsOo7eskXCsFxNADcynAGBDxK4CKfnZkcN5cdCwyl15tiLp487XKW-axXo2Oe49QlN6p5DLK-OgBdh4bZdt6jdbx7PvzoXNx2HtELjPkDvplbYz7_bwcZmHk Tbd7h1iktKMSv5f2I4zTD7CtJFGMVaN1kmTclEgh1KkQUVF91VxzB6aJ24ZFqoUsP9F_dB3cSk2kQ1TpvJlhfj00aVuLnOnD6cgaiIWQ_Dn11rqodvRdYGS7lxTmJhiwRSRkQjnIIVHT7En3k1Hymx3lIm 1EgpiQ1lVveaebj440UdN7dPAlQ6iG5XcRE3ZTGzdxAuV7adJZsI-9NNO6mOzv4bcOsYchDxhEgn_eQW0MuKhafe3-yVt8TrjvW0w4r0J0QaEC0ZxrLiIBFtnen7LVkAoQNGv2u5t-tb6fGGCDD9gFNtwK dLMC5TR-EmtGbWJehS4bVPR7URTi7czoI0S3GFSXVsqKFQRXRaVqw9uF8OZZFwRYQYKJQg2YnwqYhywfZdl2mMrLxyGeAOe5WqOzI_KeE1fros_l95FinwTbz-i-mm0mO-VDGKhnFcgk2KYQl-w71ywMgp NKXhgPmLbDFpzButkHFhF618oOc__L4ZvgFczu2YIv-_ZNc2y7NA9939cGUV45hFfzbA8JkmZIgn-80dSVhGe0cmGkeHqsETWFuryHQtA1TWnWng8VrR5GPVSFXvwj4P6IJJsBC9uPHPi8M3pRcN1gdiAq Nzq1Q4I6ht3aNpPEz16n2dJ56tMTxY30PoH_swQI4KyN4LRsrBmFUA9e4vnkA4V05xZxCdhTbmmbDvJt3Ncej06935biSR_oPgOl6A-IoW7-jZvoLxD0dkGkYQ-NrBhiO0Qxx0iCmtVME4xiSH26N3RS_K 8_odfZSSAOme1-UDqJ0soaysnxnZiE2ioA3hgku-Ns9K29ZTqBDDb68GfTg4jPzz3jeNUI8GRSuiKbqAf1et6swZHYwsfg559XIxLnWDdyRacXwic2-mA5QSGzT8uEeYEQfsxYplOGAIO4isLBiCW6hwXW ZcgC8gUpCgbdpUHaxGCxBLrYl8HY62wORzcTkOf95oEQ3oJO9GgWX_gx6LQZh-05FvUnWO_IxGd8-hzvExIZ_DqL3I139Y9vy2590oxRsI4AmdOpuS47_jD0KgDGjj_4qF2bqhvrz2HuYL1eGFVGEyT1jD RwgkvfO6qpWBfs-9-utmKAgUg0dvXj-VrZDlHNUBLuhpk8yJNUM8NHVLjpPrJpbCGbZsLw-9r0nG3U4hf8zJSnk_0npEiTsuuYK0Y_Fz1IRyzwGoYpIc7ORuBzYK3nG_AAxCIGmrI2PMO7FeKIgWaanIB0 VRnRkRf9gRv4XkzQnGPYPYFrt6zDxizklpElVK6oasZqOaVT-Oxpw6Uu5v6UR_EFzTDiycDmRqwJfxIRw_izT3BD4vP4LGtvoexq-V58z4Ob3KRUYSOYh-bO3wueZE7dd5U8R98hWqLPjgc6E76vQrpudk sUa1j_5hRt4jMJPWf3eOPHI54Npdj9Jo1tyfGaNDxw_zOIZRxCpfFunL7w4AGsgXPsRKQIhsuLvKI01W2IX5sJ-dJIbQCfRYxQn_nIM_RLxOCY1lj-aL1Qc3Yn_gECx-dUBqyKoPnTiBVF20qlpOgq-6dA -OUa31nAK240KEAIaIXerai-ucltQsnJMFEwgBJduxJPT7d2IIszqaoBJyFDavYKbgBaA2EWokZCv5ur7-UkXRvdkf9MuvjczwUKdLXIXrlyHW8QXtnJN7BtQSoU3MulPQZkcaGXdUOBHHPi-uB6TM5zzi Gypk1N7F3BvMU0yjkraKYesJy4sCbhFk0WwJOapBVQrmQvsyzcCDpdWUvGqp581Sm9YOJ2toeYRCIT63CaJqdlEEOWBXB3MLgoAJDPkMMorH1VkMPWUhGZRMxy_p9bTVMoj6Kd3-z8d70NjGRRPRmxeDli Dq5oDwSdcslGC0yepxNRgRmel5VNmugWxpbWFjrr-cURMQ0BLlWhj7YUoh_7oPw0-_8NHuSaVxtk85p7bBpXXo-9qGrNKt8nHNoPtTj0uV2q4w9ROAYFNU8iHcsjVItfV9mG3Bl4rrXUZ-MPod3u8XuTVV Fty5f8Qic3JS66LWi8vKt2SbRlto9Ai3iYSpTJa81k_IHHVQWdJ7L7waakSuiGjjKOL37f7kQRNLgtwb5lWYXjLPih39PSdHiRVT_huboQnXfhVrNMAH3eq76BBNhgxGyuZaxSqhmONee7_1LnVA5HZMx7 aAJOBkloG56jZNOvnTqbiDlgGC4neJSm-tiSIiHo4u6KONoSRLH76OL2EiVpTGRhQN3wRQkPqzvDOvUtpwJ46_SbH-cbLc-GPFuz4WlGL6dAtQ7Z9KW9TosOC79l-skMeq2EJyTPgedVoedsA01cPiXTz9 q4xYG3Hi2iBc1No-809jYSg4hzrkTUl_kv4nwOsQwU8ykQxIXr3EYgUiJ7AxLXU4ZYf3lRTXG7drE3a0n_itWG-7oOMKnfxoExo-TjC8cUJ0KUHq-ydZ3VskhhlFkSLiH-Rxs4DxzrNlRQN1q-i1O98eYN kJiOOuP4YJRSF6PF9fJX8-lQIsopvrlsx2-x4Rs_HeObEgQiW0s2RULRE16d8XaJC4ko1Rodx7oW3ZS0olIzqYT1qqhqa4oqoQjGWxkoWmRkPG6nZK4MAZkebGLNNtwyuss77HQ7UrBCidaazKhQEKFPVK dphdNY4SA=', 'amphora_id': u'ee934942-fc30-44ef-b834-e2dda0cd650f'}, 'provides': None}
According to the log, octavia failed to find the network with network id 8a588b4c-06ae-4ba2-ae74-2a8a6ed687ef, so it was unable to call nova to create the amphora virtual machine successfully.
According to the following command
root@ctl01:~# openstack network list | grep 3aa77344-3086-4d2c-aa1a-27c997c92840 | 0c1c8b1d-6515-4e5c-bba2-4a5e3ed80084 | tent_net | 3aa77344-3086-4d2c-aa1a-27c997c92840 |
We can see that this set of openstack does not have this network. In addition, there is no connection between the incoming data and the network id.
The reasons for this network id are as follows
- Added in Octavia API processing flow
- Added in Octavia controller worker
Through my keen sixth sense, there are only three ways for these two services to join the network:
- database
- Third party services
- configuration file
Let's start with the configuration file. Don't ask me what to ask. It's the simplest.
Profile troubleshooting
-
General process
- PS - EF grep target service name take a chance to find the configuration file
- Turn on grep to use grep target_id -rn config_foler or grep target_id config_file
-
Start the show
Start ps
root@cmp002:~# ps -ef | grep octavia-worker octavia 3381004 1 0 May26 ? 00:00:00 octavia-worker: master process [/usr/bin/octavia-worker --config-file /etc/octavia/octavia_manager.conf --config-dir /etc/octavia/conf.d/common --config-dir /etc/octavia/conf.d/octavia-worker --log-file /var/log/octavia/worker.log] octavia 3381139 3381004 0 May26 ? 01:25:35 octavia-worker: ConsumerService worker(0) root 4066420 4064201 0 09:38 pts/11 00:00:00 grep --color=auto octavia-worker
Locate the configuration file / etc/octavia/octavia_manager.conf And configuration folders / etc/octavia/conf.d/common and / etc/octavia/conf.d/octavia-worker
Open grep
root@cmp001:~# grep "8a588b4c-06ae-4ba2-ae74-2a8a6ed687ef" -rn /etc/octavia/ /etc/octavia/octavia_manager.conf:521:amp_boot_network_list = 8a588b4c-06ae-4ba2-ae74-2a8a6ed687ef
I'm so lucky. I found this network id in my first attempt
According to this document OpenStack Docs: Load-Balancer-as-a-Service configuration options
Configure amp_boot_network_list of networks to attach to the amphorae. All networks defined in the list will be attached to each amphora
Using ocatavia users to create a network to replace the network
export OS_IDENTITY_API_VERSION=3 export OS_AUTH_URL=http://10.8.8.11:35357/v3 export OS_PROJECT_DOMAIN_NAME=Default export OS_USER_DOMAIN_NAME=Default export OS_PROJECT_NAME=service export OS_TENANT_NAME=service export OS_USERNAME=octavia export OS_PASSWORD=trVvCUyPpHwBNfDppE7ug5we1GRvJyeU export OS_REGION_NAME=RegionOne export OS_INTERFACE=internal export OS_ENDPOINT_TYPE="internal" export OS_CACERT="/etc/ssl/certs/ca-certificates.crt" root@ctl01:~# openstack network create amp_boot_network_1 +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2020-06-19T03:20:52Z | | description | | | dns_domain | None | | id | 3ca482e2-34a5-4722-a4c5-2d7262e00945 | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | False | | is_vlan_transparent | None | | mtu | 1450 | | name | amp_boot_network_1 | | port_security_enabled | True | | project_id | cb20ced7c1ea4f43a35eb15f0ec23650 | | provider:network_type | vxlan | | provider:physical_network | None | | provider:segmentation_id | 45 | | qos_policy_id | None | | revision_number | 2 | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets | | | tags | | | updated_at | 2020-06-19T03:20:52Z | +---------------------------+--------------------------------------+
Update / etc / Octavia / Octavia / Octavia / Octavia after creating the network_ manager.conf AMP in it_ boot_ network_ List item
Then restart the Octavia worker service on all Octavia worker nodes.
Create a new loadbalancer to verify success
verification
Create a loadbalancer and view the status
POST {{endpoint}}/v2.0/lbaas/loadbalancers
X-Auth-Token: {{token}}
Content-Type: application/json
{
"loadbalancer": {
"description": "loadbalancer_created_by_gxj_for_test_purpose",
# "flavor_id": "30ad863a-25dc-425a-8ebe-e29f361cb777",
"admin_state_up": true,
"vip_subnet_id": "3aa77344-3086-4d2c-aa1a-27c997c92840",
"provider": "octavia",
"name": "loadbalancer_created_by_gxj_for_test_purpose"
}
}
root@ctl01:~# openstack loadbalancer list | grep load
| 9bae06de-d0b9-4cce-9198-389a18de9fcd | loadbalancer_created_by_gxj_for_test_purpose | 55982e2c523149158a6cc839ce2eece2 | 10.0.0.8 | ACTIVE | octavia |
Successful creation, problem solving