[go: up one dir, main page]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.16 ERROR:ServerFaultCode: NoPermission #966

Closed
Bandyman opened this issue Feb 4, 2020 · 19 comments
Closed

v1.16 ERROR:ServerFaultCode: NoPermission #966

Bandyman opened this issue Feb 4, 2020 · 19 comments
Labels
bug Type: Bug

Comments

@Bandyman
Copy link
Contributor
Bandyman commented Feb 4, 2020

Hi,
We ran into an issue this morning with not being able to create any new nodes on our vsphere.
Not a lot of usefull output with only error showing when applying a plan being
Error: ServerFaultCode: NoPermission

Setting the output to trace revealed a little more, getting the following output during the plan stage.

4261 Error: ServerFaultCode: NoPermission
4262 2020-02-04T12:45:39.115Z [DEBUG] plugin: plugin process exited: path=/build/terraform/terraform-windows-vm/projects/.terraform/plugins/linux_amd64/terraform-provider-vsphere_v1.16.0_x4 pid=202
4263 Error: ServerFaultCode: NoPermission
4264 2020-02-04T12:45:39.115Z [DEBUG] plugin: plugin exited
4265 Error: ServerFaultCode: NoPermission
4266 Error: ServerFaultCode: NoPermission
4267 Error: ServerFaultCode: NoPermission
4268 Error: ServerFaultCode: NoPermission
4269 Error: ServerFaultCode: NoPermission
4270 Error: ServerFaultCode: NoPermission
4271 Error: ServerFaultCode: NoPermission
4272 Error: ServerFaultCode: NoPermission
4273 2020-02-04T12:45:39.116Z [DEBUG] plugin: plugin process exited: path=/builds/terraform/terraform-windows-vm/projects/.terraform/plugins/linux_amd64/terraform-provider-vsphere_v1.16.0_x4 pid=189
4274 2020-02-04T12:45:39.116Z [DEBUG] plugin: plugin exited
4275 Error: ServerFaultCode: NoPermission
4276 ERROR: Job failed: exit code 1

Terraform Version: 0.12.18
vSphere Provider Version : 1.16.0

Our account according to IT has full admin privalges on vsphere, reverting back to vsphere plugin version 1.15.0 and hard locking to that version fixed it for us.

Let me know if you need more information and I try and help
Thanks,
Tristan

@f4nha
Copy link
f4nha commented Feb 4, 2020

Same here,
Failed on 1.16 works fine on 1.14 and 1.15

Error: disk.0: validation failed (ServerFaultCode: NoPermission)

  on machines.tf line 11, in resource "vsphere_virtual_machine" "tftest":
  11: resource "vsphere_virtual_machine" "tftest" {

Thanks

@JosephHobbs
Copy link

Seeing the same issue here as well...

@8uachaille
Copy link
8uachaille commented Feb 4, 2020

We are experiencing the same issue with v1.16 - Please see our error in context of DEBUG log

2020-02-04T15:15:37.988Z [DEBUG] plugin.terraform-provider-vsphere_v1.16.0_x4: 2020/02/04 15:15:37 [DEBUG] VM “/“path/to/our/template/ISO found for UUID "4201b507-7907-3b1a-55d3-cdef9f4264cd"
2020-02-04T15:15:38.007Z [DEBUG] plugin.terraform-provider-vsphere_v1.16.0_x4: 2020/02/04 15:15:38 [DEBUG] queryAssociatedProfile: Retrieving storage policy of server object of type [virtualDiskId] and key [vm-1092382:2000].
2020/02/04 15:15:38 [ERROR] root: eval: *terraform.EvalDiff, err: disk.0: validation failed (ServerFaultCode: NoPermission)
2020/02/04 15:15:38 [ERROR] root: eval: *terraform.EvalSequence, err: disk.0: validation failed (ServerFaultCode: NoPermission)
2020/02/04 15:15:38 [TRACE] [walkPlan] Exiting eval tree: vsphere_virtual_machine.vm
2020/02/04 15:15:38 [TRACE] dag/walk: upstream errored, not walking "meta.count-boundary (count boundary fixup)"
2020/02/04 15:15:38 [TRACE] dag/walk: upstream errored, not walking "provisioner.local-exec (close)"
2020/02/04 15:15:38 [TRACE] dag/walk: upstream errored, not walking "provisioner.remote-exec (close)"
2020/02/04 15:15:38 [TRACE] dag/walk: upstream errored, not walking "provider.vsphere (close)"
2020/02/04 15:15:38 [TRACE] dag/walk: upstream errored, not walking "provisioner.file (close)"
2020/02/04 15:15:38 [TRACE] dag/walk: upstream errored, not walking "root"
2020/02/04 15:15:38 [DEBUG] plugin: waiting for all plugin processes to complete...
2020-02-04T15:15:38.225Z [DEBUG] plugin.terraform: remote-exec-provisioner (internal) 2020/02/04 15:15:38 [ERR] plugin: plugin server: accept unix /tmp/plugin930112039: use of closed network connection
2020-02-04T15:15:38.225Z [DEBUG] plugin.terraform: remote-exec-provisioner (internal) 2020/02/04 15:15:38 [DEBUG] plugin: waiting for all plugin processes to complete...
2020-02-04T15:15:38.225Z [DEBUG] plugin: plugin process exited: path=/home/terraform/bin/terraform

@arsiesys
Copy link
arsiesys commented Feb 4, 2020

Hello,

Same issue here.

After debug, it's related the following change:
12e2fc9

Could we know which access/role name is missing to be able to fix it ? :p
Thanks!

@aareet aareet added the bug Type: Bug label Feb 4, 2020
@aareet
Copy link
Member
aareet commented Feb 4, 2020

Thank you for filing this issue - we're investigating the problem

@bill-rich
Copy link
Contributor

I'm working on tracking down the potential causes of this issue. There are a few data points I could use that would help make sure I cover all the cases.

  1. What vCenter/vSphere version are you using?
  2. Does the user Terraform is running as have "Profile-driven storage" permissions at the vCenter level?

Thanks, and I'll provide updates shortly.

@arsiesys
Copy link
arsiesys commented Feb 4, 2020

Hello @bill-rich,

We are running in Vcenter 6.5.

The user running terraform had some specific RW access on ressources pools/Datastore and was running fine in 1.15. Also, the user had a global read only access on the vcenter.
However, it seems that the global read only do not cover the profile-driven storage.
With the complementary access "profile-driven storage view", it work!

I guess it could be good to document (or catch the error and print a detailed output) it as we will not be the only ones to get impacted :p.

@vkmellon
Copy link
vkmellon commented Feb 5, 2020

The same we have: err: disk.0: validation failed (ServerFaultCode: NoPermission)

@kavson
Copy link
kavson commented Feb 5, 2020

Also just started getting this error:

Error: Error running plan: 2 errors occurred:
* module.rhel.vsphere_virtual_machine.vm: 1 error occurred:
* module.rhel.vsphere_virtual_machine.vm: disk.0: validation failed (ServerFaultCode: NoPermission)

* module.rhel.vsphere_virtual_machine.vm: 1 error occurred:
* module.rhel.vsphere_virtual_machine.vm: disk.0: validation failed (ServerFaultCode: NoPermission)

@8uachaille
Copy link

Previously:

We were experiencing the above (v1.16 ERROR:ServerFaultCode: NoPermission #966) error

We were running Terraform v0.11.11 with v1.16 vsphere provider against vCenter 6.7 / ESXi 6.5

We got the following error: err: disk.0: validation failed (ServerFaultCode: NoPermission)

Now:

I tried to work-around this problem I still get the following error running terraform plan

 upgraded Terraform to v0.12.20 
 ran terraform 0.12upgrade
 Allocated Profile-driven storage (view) privilege to Terraform-related user role

 Error: disk.0: validation failed (ServerFaultCode: NoPermission)

   on config.tf line 32, in resource "vsphere_virtual_machine" "vm":
   32: resource "vsphere_virtual_machine" "vm" {

   [terraform@nohost ]$ /var/tmp/terraform --version
   Terraform v0.12.20
   + provider.vsphere v1.16.0
   [terraform@nohost ]$

I would like to have a working approach for vCenter 6.7 if possible

It would also help to know how to select a specific vSphere provider version say v1.15.0

I tried the following stanza which seemed to agree with the provider documentation at
https://github.com/terraform-providers/terraform-provider-vsphere

provider "vsphere" {
version = "~> 1.15"
user = “not”
password = “working”
vsphere_server = “server”
allow_unverified_ssl = true
}

However, my terraform run continues to use 1.16.0

[terraform@nohost]$ /var/tmp/terraform init
Initializing the backend...
Initializing provider plugins...

  • Checking for available provider plugins...
  • Downloading plugin for provider "vsphere" (hashicorp/vsphere) 1.16.0...

I looked for a state file to determine whether I needed to purge that but there isn't one in the pwd after the terraform plan nor is there one anywhere else on the host

@jgrancell
Copy link

It would also help to know how to select a specific vSphere provider version say v1.15.0

I tried the following stanza which seemed to agree with the provider documentation at
https://github.com/terraform-providers/terraform-provider-vsphere

provider "vsphere" {
version = "~> 1.15"
user = “not”
password = “working”
vsphere_server = “server”
allow_unverified_ssl = true
}

You need to specify the version string correctly:

provider "vsphere" {
  version = "< 1.16.0"
  ...
}

The key being the < symbol which means you want a version less than 1.16.0. By using ~> you're specifying you want a release equal to or greater than 1.15, but below 2.0.

@8uachaille
Copy link

That worked - thanks for your help Josh

@bill-rich
Copy link
Contributor

Thanks for testing that @arsiesys!

For everyone still experiencing this issue, it looks like is is due to new permissions being required for the addition of SPBM support in v1.16.0. Please check that the user Terraform is running as has "Profile-driven storage" permissions at the vCenter.

I will get the changelog updated with notes about the additional permissions.

@8uachaille
Copy link

Still fails with v0.12.20 and v1.16.1

$ egrep -i 'terraform|1.16' terraform.log | head
2020/02/19 09:24:00 [INFO] Terraform version: 0.12.20
...
2020/02/19 09:24:00 [DEBUG] fetching provider location from "https://registry.terraform.io/v1/providers/hashicorp/vsphere/1.16.1/download/linux/amd64"
[terraform@terraform ece02.vh.iot.ed.ac.uk]$

Error:
WARNING:
There was an error performing post-clone changes to virtual machine "/MY Datacenter/vm/YY/ Servers/my.f.q.d.n":
error processing disk changes post-clone: disk.0: ServerFaultCode: NoPermission: RESOURCE (vm-1215521:2000), ACTION (queryAssociatedProfile): RESOURCE (vm-1215521), ACTION (PolicyIDByVirtualDisk)
Additionally, there was an error removing the cloned virtual machine:
error destroying virtual machine: ServerFaultCode: Permission to perform this operation was denied.

The virtual machine may still exist in Terraform state. If it does, the
resource will need to be tainted before trying again. For more information on
how to do this, see the following page:
https://www.terraform.io/docs/commands/taint.html

If the virtual machine does not exist in state, manually delete it to try again.

on config.tf line 35, in resource "vsphere_virtual_machine" "vm":
35: resource "vsphere_virtual_machine" "vm" {

@aareet
Copy link
Member
aareet commented Feb 19, 2020

@glenfiddich have you ensured this - Please check that the user Terraform is running as has "Profile-driven storage" permissions at the vCenter.?

@stevenklar
Copy link

Upgrade to 1.16.1 and provide the mentioned permissions "Profile-driven storage" fixed it for us.

@Ekallatum
Copy link

Plugin version 1.16.2.

It's seems that it is necessary to set "Profile-driven storage" policy at the root group of vcenter server.

Fixed for us.

@aareet
Copy link
Member
aareet commented Mar 11, 2020

Closing this issue - please create a new issue if this recurs in current or future versions of the provider.

@aareet aareet closed this as completed Mar 11, 2020
@ghost
Copy link
ghost commented Apr 18, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

@ghost ghost locked and limited conversation to collaborators Apr 18, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Type: Bug
Projects
None yet
Development

No branches or pull requests